There continues to be confusion about the role of Applications DBAs in supporting the E-Business Suite (EBS). We still encounter organizations that believe that any Oracle DBA can easily transition over to being an Applications DBA (it is still a database after all!). For them, EBS is seen as just another commercial-off-the-shelf (COTS) application with some initial configuration work.
This assumption rarely works out and we often find that those new Application DBAs (APPSDBA) struggle to understand their new role. Actually, we sometimes find large gaps in the maintenance of the E-Business Suite environment and less than stellar Applications performance because the new APPSDBA has no idea what they don’t know. Even if they are lucky enough to get some training or good mentoring, they may still come back to an environment carrying a long list of responsibilities while not necessarily understanding how it all fits together.
Traditional Oracle DBAs can also be hampered by their natural focus on monitoring database performance, ensuring backups, resolving database issues, installing releases and improving database performance. These are important tasks but in the EBS environment they are only a portion of the ongoing needs for the environment. The appropriate solution for an E-Business Suite backup is much different from a standard database backup, since it must include backing up the entire E-Business Suite, including database and code. Often it is assumed that the Application Tier is relatively static and can be backed up anytime, but that may not be the case.
The new APPSDBA also needs to configure monitoring tools and backups, understand and control release management, and proactively evaluate potential performance enhancements for the Application Tier (Forms, Reports, Java, Discoverer and Apache), and understand how those changes will impact EBS clients. When that is done they need to know how to properly clone for testing (a more complicated task because of the complexity of the E-Business Suite environment), monitor performance of the JVMs, forecast application server/storage requirements, sort out and apply relevant patches, and maintain application security at the Operating System and database levels. The nuances of the E-Business Suite, it turns out, require both a broad and deep understanding of not just the database, but the tools that are used with the database.
The result is that in smaller environments, one properly trained and experienced APPSDBA might keep on top of it, but that is rare in medium and larger environments. Some Applications DBAs just do the best they can and never get around to patching or maintenance. As long as the end-users are not asking, they can keep everything working, even though the backlog of future work may be growing rapidly.
So why does the APPSDBA role differ so much from that of a traditional DBA? At a high level I think it is because of the close integration of the database and applications, and the introduction of additional technologies that are required.
The close integration of the Oracle database and the E-Business Suite leads to high levels of dependencies. These many dependencies create limits on what can be done and when. Tuning, configuring, patching and upgrading always require plenty of review before proceeding.
The area of application patching is enough to confuse most people (including me!). First, there are multiple types of patches and a whole dictionary of patching terms (RUP, Emergency, Mini-Packs, Consolidated Updates) whose definition may change over time.
Then the APPSDBA has to read the Release Notes and decide where to get started. Many APPSDBAs then download the patch and examine the detailed steps of the patch just to make sure they understand what is really happening behind the scenes. They try to determine beforehand whether the patch will fix the problem users are experiencing, or cause some other problem. But what if Oracle requires the patch, but the users are experiencing no issues? Will applying it create a whole set of new issues that will cause sleepless nights?
In each case, the APPSDBA is challenged to understand new technologies but also work with an environment that has some technologies that are over 5 years old. A good example with Release 11i (as well as Release 12) is that there are multiple Oracle Homes to support the technology changes that have occurred since its introduction in 2004.
If you recall, a new release of the E-Business Suite is normally eligible for Premier support for 5 years by Oracle. During those five years there will be tens of thousands of patches released just for the Applications alone. While it is getting easier to apply them, it’s not getting easier to decide when, how and what the impact will be. And the windows for planned downtime continue to narrow.
Upgrading the Applications or database isn’t plug-and-play either. Upgrading to new versions of the database requires thorough testing and practice to ensure that the Applications perform correctly under the new database. Also, just because a new version of the database is available doesn’t mean E-Business Suite users can begin using it. Oracle RDBMS Version 10.2.0.4 provides a good example – it’s available now, but is not yet certified to run with the E-Business Suite.
Also, moving to the latest certified version of the RDBMS isn’t enough – you need to thoroughly read the documentation to determine if init.ora parameters need to be changed or if Oracle has made other significant changes. One example is Cost Based Optimization. Over the years, Cost-Based Optimization has slowly permeated the E-Business Suite database, requiring different settings depending on the database used. On the 9i database, SYS and SYSTEM objects can optionally be analyzed, but on 10g, it is mandatory to analyze those objects. That’s a detail that only comes out by reading through the manuals.
How about Applications upgrades? Upgrading to new Applications releases is a planned multi-month process (often 6-12 months) requiring a camping permit on Metalink as a prerequisite. Often you find yourself opening multiple Service Requests just to get through the process.
The additional technologies are not just limited to the database. Java is a key component of the Application Server and it has regular new releases. At the client level, many organizations are in the process of moving away from Oracle’s JInitiator and replacing it with the Java Plug-In. Should they use Java 1.5 or 1.6? How do they regulate what the user’s desktop will choose if the user has multiple versions of Java installed? Are there tools for tuning Java?
Suffice it to say that in any one calendar year there is some version of the EBS technology that is being retired while new ones are introduced. As the Applications DBA, you are responsible for ensuring that it happens without impacting the critical business processing of the organization.
Now, if I had more space I’d jump into the need for a separate Applications System Administrator and Workflow Administrator, two other important roles in the EBS environment. But for now, I’ll leave you with these questions: What are your thoughts about APPSDBAs? How does your company handle the challenges of this job? Let me hear your thoughts.