New Additions to the UCI Degreeworks Team

Nancy Gasparotti has retired. She was a great worker and an awesome Degreeworks resource. We had an great Sock Hop themed retirement party. I am very happy for her because she is loving her new retirement life.

She gave us plenty of warning of her retirement which helped us acquire and train new scribers. The new scribers are:

Mark Fonseca, ifonseca@uci.edu

and

Jennifer Prescott, jprescot@uci.edu

They have been very well trained by Nancy. Feel free to ask them scribe questions or drop them a line.

Derrek

One thing fast is better than many things slower.

Yes, it been a while but I'm going to make up for it right now. Not only did we move our production account from 772 to 405 we switched CPU's we're now running on RedHat. Why? Money and performance, mostly for performance.

Before:

7.7.2
Sun V880 - 8 x 1.2 GHz
RAM - 16GB
Hosting: Database, webserver, DGW test and prod.
Performance: 15-20 second audits.


After:

4.0.5

2 x Dell T100 - Intel Quad 3.0 GHz
RAM - 4 GB
Hosting: Prod account only
Performance: 1-3 second audits.

Yes, that is a HUGE jump. My final grade batches went form 3 days to less than one day.
I could run it faster but 405 has some DGWCPUCOUNT issues that prevent me from running
simultaneous batches. I've got a SR for it.

The best part of this migrations is that my new Dell servers only cost me $1000 EACH!!

What happened to the Sun T5220's? I was pretty disappointed. I was hoping to take advantage of it's award winning multi-threading but the problem was that Degreeworks is not a multi-threaded app. I tried EVERYTHING to make the T5220 to work for us. (that's why I hadn't posted in a while.) The ONLY advantage of the T5220 at 1.6 Ghz is that you can batch faster by setting the DGWCPUCOUNT to like 10 and have 10 batch jobs running at the same time.

Listen closely now because this is a proven theory.

CPU speed is the key to increasing performance.

There. I said it. Yes, throw money at it but not a lot. Since DW is single threaded it will never take advantage of multi-threaded processors when it comes to dynamic web audits. So why have a server that does multi-threading at a slow speed (Sparc 1.6 GHz) when the app will never take advantage of it? So I bought a server that does one thread really really fast (Intel 3.0 GHz).

*(Note: Sungard also recommends a processor with at 3.0 Ghz and higher)*

I know about the WEB09M03 and, in my opinion, it does absolutly nothing. In fact, I think it makes things worst. It basically uses one thread and to jump between several requests. This is an old configuration which helped with managing cobol licenses in the old versions (772 and earlier).

Those T5220's didn't go to waste. One is used as a primary database for Degreeworks and the other is used for our test environment.

Note that separating the database from the app also made a huge difference. There is a lot of I/O going on. Now Oracle (a major resource hog) and DW don't have to compete for resources any more. This is one thing every site should do.

This layout is perfect for us right now. I'm on the verge of releasing what-if to students because it's performing so well. We never could give it to the students because 20,000 students hitting what-if at a rate of 20-second audits on a single thread would give me nightmares.

I could go on and on about this but this is enough to chew on for a while. More posts to come. I promise.

The only thing constant is change.

Sorry I haven't posted much in the last couple of weeks. I was a little swamped with DW4.0.1 fixing and debugging.

DW4.02 is here and it is packed with new stuff. I consider it more of a mini release instead of a patch because it's got a lot of cool new things. I'm having minor issues with the installation package right now but I will give a full report soon which will include time line and adoption considerations.

For standalone users: The BRIDGE has changed once again. (I think twice in just this release of DW4) RAD-DEGREE-DTL and RAD-SCHOOL-DTL has been phased out (although it is backwards compatible). The replacement is RAD-GOAL-DTL and RAD-GOALDATA-DTL. It allows for unlimited calling of major/minor/program blocks.

I've only glanced at it and it a little confusing. I'll have to experiment.

Stay tuned. I'll give my review soon.

DW4.0.2 - On the way!!

I just closed a Service Request and it mentioned that DW4.0.2 is going to be delivered at the end of the month. So start planning your upgrade implementation!!

We notified SGE about a month ago that we wanted to move production in DW4.0.x in March and they have been very responsive to our needs. Most of our problems and defects have been answered.

I have a lot of confidence that we will have some sort of mirrored production version of DW402 and 772 at the end of the month. That way our users can use both versions and ease into the DW40x. I did this setup a couple of years ago and it worked out fine.

Database indexing

Problem: Audits getting slower over time but nothing has changed.
Solution: Re-index your database audit tables.

At UCI, we use In-progress classes and update them on a daily basis. With 30,000 students in DW, a lot of database records get changed, thus database indexing becomes overwhelming.

DW and Oracle doesn't know when or how to re index the audit tables so it would be wise to add this to your quarterly maintenance. If I don't re index, our audits take twice as long to process.

The tables - dap_audit_dtl, dap_audtree_dtl

The indexes - for dap_audit_dtl: (via xsql, select index_name from dba_ind_columns where table_name='DAP_AUDIT_DTL';)

for dap_audtree_dtl: (via xsql, select index_name from dba_ind_columns where table_name='DAP_AUDTREE_DTL';)

Two options in re indexing:

A) Simply rebuild the index in xsql:

1. login to the account.
2. login to xsql.
3. type the rebuild index command for each of the indexes: "alter index DAP_AUDIT_DTL_A rebuild unrecoverable;"
4. Do this for each index. (DAP_AUDIT_DTL_A, DAP_AUDIT_DTL_B... etc.)

B) Rebuild the index in a new tablespace (recommended for very large FTE's):

1. Same as above but add the tablespace command: "alter index DAP_AUDIT_DTL_A rebuild tablespace (tablespace name) unrecoverable;"

Option B recommended by Oracle DBA for very large tables because instead of using temp space to create the new indexes, it just builds them in the new tablespace. Plus this is a much faster process.

Our I alternate between two tablespaces that are 6 GB large just for dap audit indexes. Yes, that is very large. Our tablespaces just for audits are 20 GB.

** Please consult with your IT staff at your site before experimenting with this. ***

DW4.0.1 Testing (Update)

Recommendation: Wait til the release of DW4.0.2. Sungard didn't tell us when it would be released.

In DW401, 2 out of 4 critical SR's went to defect. Here's how it went down:

1) Program Block not being pulled in. - Defect - we got an updated dap32s.c. It works well. The SR states that the fix won't be delivered until 4.0.2. No date yet.

2) Exceptions un-hooked - Defect - Major show stopper. I go into more details here. Sungard decided to modify the old 772 database entries with a sql script and continue with the new DW401 format. I don't know how they are going to deliver this. I just got the fix Friday. I assume it will be in 4.0.2.

3) "Apply here" applies something else - head scratcher - We couldn't reproduce it.

4) Custom Dtls - ucx table correction - Back in 772, we could be lazy and not specify element numbers. Now in DW401 you have to specify the element numbers. I put in R323 for the data element and R322 for the Edit Element 1. This is stated on page 81 on the DGW Technical Guide UCX.

Again, we are all different clients. We're old clients moving from 772 to DW401. If you are a new client starting in DW401, these may not be issues.

We have other Service Request open but are fairly minor. I'll post them if they make it to Defectville.

Exceptions falling with 772 to DW401 migration.

Our #2 show stopper for going live with DW4.0.1 is showing to be a major problem. Identified and confirmed by us and DW development, all exceptions will fall off if a site migrates from 7.7.2 to DW4.0.1.

Explanation:
The dap_exc_buffer is database field that holds the course info that the exceptions is using. In 7.7.2 it looks like this:
After the database conversion step in the upgrade the field and data stays the same. But when you run a new audit, the auditor will read it starting two bytes off. It's now looking for, "OG 13" as the discipline and "0" as the number. I confirmed this by entering a exception in the new DW4.0.1 account exactly like the previous one and queried the database table and compared:
Very different from before. A possible COBOL to C conversion problem.

As of right now, we haven't received a solution or a way to proceed. Here's my guess solutions: 1)Change the database entry for ALL the 772 to read old entries. 2) Release a hot fix to go back to the way the buffer is made. 3) Write code to have the two different layouts be valid.

I'll let you know if I get the fix before you do.