Deployment on Dynamo
What problems do system administrators encounter in the field?
In order to answer this we need to define the role with respect to the work flow on the site. Below is an outline of a generic Dynamo site (comments welcome).
In this case the site/system admin (SA) has responsibility for the Staging and Production systems. QA shares responsibility for the Staging environment since they will be doing level 3 testing there, but the SA is responsible for the code pushes since this is where they practice for production.
So what does this mean for the SA? Well let's make the admittedly outrageous assumption that the software is functionally complete at the time it's approved. What problems may the SA face?
The root of problems for the SA lie in the differences between the testing environment and the production environment, key amongst these being:
- Production database
- Production hardware
- Addition of the web server
- Clustering
- Increased load
- JVM and Java environment
- Code Error / Transfer Problems
Within these areas we aim to impart on our students a set of skills which can
be used to detect,identify and isolate most Dynamo problems.
We also need to keep the labs relatively simple to administer and execute for
trainers and potentially e-learning. Therefore, where possible, the games should
be playable on a standard install.
The target skill set is as follows:
- Stack trace generation and analysis
- Log file analysis
- Site response (user and URLHammer)
- The VMSystem component
- The Performance Summary component
- Component by component debugging
Common Deployment Issues
Production Database
- Wrong database URL or password (Exception) (error.log)
- Column size data truncation (Exception, SQL error) (error.log)
Production Hardware
- Filename case sensitivity - This.txt is not the same as this.txt
- Windows coded filename in component property - C:\ATG\Dynamo5.1\home\... (error.log)
Web Server
- Content published to the MEDIA directory from the database is not on the web servers
- JSP pages are erroneously being served by the web server instead of Dynamo
Clustering
- Incorrect global scenario server configuration
- Incorrect load management configuration
Increased load
- Thread deadlocking shows up under load (VMSystem, site response, Thread dumps, stack traces)
- Loss of database connections over time (debugging, site response, warning.log)
JVM and Java environment
- Database drivers not found (error.log)
- Java out of memory error
Code Error / Transfer Problems
- Component not on CONFIGPATH, but is on machine (site response, error.log)
- Slow page (Performance Summary / Monitor)