One thing fast is better than many things slower.

Yes, it been a while but I'm going to make up for it right now. Not only did we move our production account from 772 to 405 we switched CPU's we're now running on RedHat. Why? Money and performance, mostly for performance.

Before:

7.7.2
Sun V880 - 8 x 1.2 GHz
RAM - 16GB
Hosting: Database, webserver, DGW test and prod.
Performance: 15-20 second audits.


After:

4.0.5

2 x Dell T100 - Intel Quad 3.0 GHz
RAM - 4 GB
Hosting: Prod account only
Performance: 1-3 second audits.

Yes, that is a HUGE jump. My final grade batches went form 3 days to less than one day.
I could run it faster but 405 has some DGWCPUCOUNT issues that prevent me from running
simultaneous batches. I've got a SR for it.

The best part of this migrations is that my new Dell servers only cost me $1000 EACH!!

What happened to the Sun T5220's? I was pretty disappointed. I was hoping to take advantage of it's award winning multi-threading but the problem was that Degreeworks is not a multi-threaded app. I tried EVERYTHING to make the T5220 to work for us. (that's why I hadn't posted in a while.) The ONLY advantage of the T5220 at 1.6 Ghz is that you can batch faster by setting the DGWCPUCOUNT to like 10 and have 10 batch jobs running at the same time.

Listen closely now because this is a proven theory.

CPU speed is the key to increasing performance.

There. I said it. Yes, throw money at it but not a lot. Since DW is single threaded it will never take advantage of multi-threaded processors when it comes to dynamic web audits. So why have a server that does multi-threading at a slow speed (Sparc 1.6 GHz) when the app will never take advantage of it? So I bought a server that does one thread really really fast (Intel 3.0 GHz).

*(Note: Sungard also recommends a processor with at 3.0 Ghz and higher)*

I know about the WEB09M03 and, in my opinion, it does absolutly nothing. In fact, I think it makes things worst. It basically uses one thread and to jump between several requests. This is an old configuration which helped with managing cobol licenses in the old versions (772 and earlier).

Those T5220's didn't go to waste. One is used as a primary database for Degreeworks and the other is used for our test environment.

Note that separating the database from the app also made a huge difference. There is a lot of I/O going on. Now Oracle (a major resource hog) and DW don't have to compete for resources any more. This is one thing every site should do.

This layout is perfect for us right now. I'm on the verge of releasing what-if to students because it's performing so well. We never could give it to the students because 20,000 students hitting what-if at a rate of 20-second audits on a single thread would give me nightmares.

I could go on and on about this but this is enough to chew on for a while. More posts to come. I promise.

Sun versus Red Hat

versus

7.72DW2.0 was the first release of Degreeworks with the Red Hat as an option. Before I bought my hardware I had the opportunity to test both environments. My Review:

Red Hat:

Pros: Cheaper. Much Faster CPU's (up to 3.0 Ghz), thus faster audits.
Cons: Limited to only Intel processors. NO AMD's! Less cores.

Sun:
Pros: Sun hardware allows for more core/multi-thread processing, thus more batches/simultaneous audits.
Cons: Expensive. Slower CPU's (1.0 - 1.66 GHz).

That's basically what it boils down to. They both run DW exactly the same. Obviously the RH server ran the audit 2-3 times faster but only has a 4 core limitation. Sun processors go up to 8 cores.

I tried to run a Sun server and a RH server connected to the same database account (hybrid server) and it didn't work because of steno. Eventually this will be a option once steno has been phased out of DW. 1-2 years?

I chose the Sun because eventually we are going to need to run more batches for the Curriculum Planning Assistant. I will blog about this soon. It's has some costly needs.

Hardware... a key to DW success.

What we have:

Two Sun Sparc T5220 with 4 core threads.
4 GB of RAM
292 GB of Storage.

It's Sun's base model. This is sufficient for our needs and it fit our budget at the time of purchase.

We named them sole and luna, it's Italian for sun and moon. One server acts as the primary server and the other acts as an axillary server for student what-if processing and CPA batching.

Our previous inadequate server was a Sun Sunfire V880. It had served Degreeworks for 8 years and wasn't good at all. We were only able to manage because I setup a queue so that DW could run no more than 5 simultaneous audits. Any more than 6 audits at the same time would make the machine run like a turtle in molasses.

Now many clients have been successful and happy with this server but I couldn't wait to take it out back and beat it with a Louisville Slugger.

One reason why we weren't happy was the poor I/O bus speeds on the V880. We set up a software RAID (hardware RAID isn't available for this server) on the drives and writing takes up to many resources. Thus, a longer wait time.

The other big problem that might be key for other institutions of similar size is the complexity of our scribing. Our requirements for GE is filled with group rules. For many small college clients, an audit will process in about less than 5 seconds. For us, it's about 15-20 seconds. This may not seem like a lot, especially for other sites with less than 8,000 FTE's, but we have 28,000 FTE's.

On a side note, we're not the slowest audit. I think Boston University is running at 25-30 second audits for 30,000 students. They wrote programs similar to us to manage this issue.

We had DW consultants look into our situation and they couldn't help us. We were stuck with this setup for about 8 years.

Enough with our history. I know Sungard never endorses a machine and gives you a hand out on what they recommend. I've given them permission for new clients similar in size and capacity to contact me to help them with their hardware decisions. This is what I send.

My Suggestions for Hardware Purchase:

1) Faster CPU = faster audits

2) More CPU's or threads = faster batches (from the V880 to T5220, a new grade processing goes from 48 hours to 6 hours.) This will be important if you plan to use the Curriculum Planning Assistant.

3) More memory = More accounts. We only have room for Test and Prod, we like to get more resources for a Dev account. This basically has to deal with Oracle and the need to create a db instance for each account. If you host the DB on a different machine, you won't have this problem.

Having more accounts is a definite recommendation. Especially for debugging DW updates. (And you will debug the new updates. I'll post details later.)

4) Maximum hard drive space!!! You won't believe how much space the product takes. Although the product needs about 5-7GB of space, your database is going to be enormous. Our Prod account is roughly 50-60 GB. Yes, no joke. Half for just audits and the other half for the CPA warehouse. We have 28K students and growing.

Retention is an issue but I've got a work around to store the audit "page", not the audit tree, in a separate DB table. This saves time and resources for batch auditing.

Good luck finding hardware!