Integrating CA Software Change Manager with a Java EE application

CA’s Software Change Manager is a tool that we use to manage software and documents.  Although much of our development staff is using GIT, many of our engineers still use SCM (or Harvest as it used to be known).  The primary way our web applications use Harvest is as a version controlled repository for documents.  Our web apps can link directly to the latest versions of documents and can even allow our users to checkout and modify documents through the web.  I’d like to share some of the techniques we’ve used to build a relationship between a Java EE application apps and Harvest.

The obvious choice for using Harvest with java is the JHSDK (Java Harvest SDK).  But through trial and error, I’ve learned that JHSDK fails in Java EE because it’s classes are not thread safe.  At the CA World conference a few years ago, I had the privilege of consulting with the father of Harvest (can’t remember his name) about this problem.  He said that I have two options, make the API thread safe or just make system calls to the command line interface (Harvest CLI).  Not knowing exactly how to make the API thread safe, I chose the latter.  It has never failed us.

Things to consider when designing a Harvest-related web app:

1.  The server on which your app server resides must have a Harvest client installed.   You’re essentially creating an automated Harvest user in your web app.

2.  Because you’re using a CLI instead of JHSDK, you have to retrieve errors and exceptions by reading logs.  Each Harvest command creates its own log file on the server.  So you have to manage log files in real time.  We create a new log file with a new random name with every command.  After the command runs, we check the log messages so that we can send the messages (including errors) back to the browser.  And finally, we delete the log file.  This is handled differently for asynchronous calls (see #5).

3.  Each Harvest command must contain user credentials.  When the user logs in, you could capture his/her username and password, but this isn’t very secure.  Ultimately, you want to use an auth file on the server.  This can be generated with a command using the username and password one time and never again (unless the password changes).  You name the auth file after the user name and then you can reference it anytime you need it.  The svrenc command looks like this:

String[] command = new String[11];
int i = 0;
command[i++] = “svrenc”;
command[i++] = “-f”;
command[i++] = userName+”.auth”;
command[i++] = “-usr”;
command[i++] = userName;
command[i++] = “-pw”;
command[i++] = password;
command[i++] = “-dir”;
command[i++] = siService.getAuthRootPath();
command[i++] = “-o”;
command[i++] = log;

4.  Building commands can be error prone if it’s done in one big String.  Fortunately, Java’s Runtime class takes a String array.  It’s best to build your commands this way. (see above example for building a command)

5.  Commands can be run asynchronously for long-running processes by putting the command into a thread.  As the thread runs it writes the progress of the log file output into the database and the client polls it with AJAX calls.  That way you can show progress on the process.

6.  When the user needs to view a file, it’s easier and quicker to use SQL rather than the hco command to get a blob and stream it out to the browser

7.  When the user needs to checkout a file (hci command), you’re checking it out to the server’s file system then streaming a copy of that file back to the web browser.

8.  To check in a file, upload the file through the web browser to the exact spot where it was checked out to, then run the hco command on it.

9.  Finally, use the documentation.  It comes with your installation and is called “CA Software Change Manager Command Line Reference Guide”.  Everything you can do with the SCM client can be done via the CLI, and therefore can be done in a web app.

Integrating your web apps with CA SCM can be a very powerful asset to your users.  We allow users to list, view, and manage files, promote/demote packages, edit forms, create comments, and approve/deny packages.  We had hoped that CA would be a decent web version of SCM, but it never happened, so we built the parts that we need.  We’ve been very successful, and using CLI calls has been very reliable.

Step 1: Setup a new Life Cycle

CA defines a Life Cycle as:

…the set of all the stages (states) a change or release goes through from identification to completion; it defines a model for a particular development and maintenance process.

Anyone who’s been with this product for a long time will inevitably chuckle to hear the term Life Cycle because it’s the third name for this feature.  The evolution is  Environment–>Project–>Life Cycle.  In the database, you can still see the evolutionary evidence of Environment in the table name HARENVIRONMENT.  You can also see the evidence of the product’s former name:  Harvest.  🙂

Regardless of what you call it, it’s a damn good thing.  This is what differentiates CA SCM from a version control system.  It is a Software Change Management system.  You cannot change source without a Package (CA SCM’s representation of a software change unit of work or project task), and you cannot have a Package without a Life Cycle.   It’s up to you just how much accountability and control you want for you Life Cycle, but this is a tool that really wants you to operate in a well-defined, well-documented, repeatable process.

You can start from scratch or you can use another Life Cycle as a template.  For the sake of repeatability, I chose to use an existing Life Cycle as a template.  CA SCM comes with a library of very useful templates from which to choose.   I chose the Release Model for this particular software release.

Release Model Life Cycle Template

Release Model Life Cycle Template

It has four States:  Development, Test, Release, and View SnapShots.  Each state has a list of pre-built processes.   Also, notice that it has a set of Data Views.  A view is how you isolate change in states.   For example, the Test view shows the state of the code in the Test State.  You will not see code that’s still in the Development State in the Test View.

To make this template an Active Life Cycle:

  1. Select “Copy To …” from the context menu for the template

    Copy To

    Copy To

  2. Name the new Life Cycle.   You should give it the name of the software release you are about to begin.

    Copy Project Dialog

    Copy Project Dialog

  3. Click OK!

By default, rights and access within this Life Cycle are controlled by two User Roles:  Dev Manager and Developer.   You must assign team members one of these roles for them to be able to use this Life Cycle.  You can also choose to create your own User Roles for more fine grained and project-specific control.

So, we have a life cycle.  These out-of-the-box templates are  very well-documented, in the Administrator Guide, so I won’t go into the details.  But I will say that this is a fairly sophisticated release management process that can be extended to become a full-blown continuous integration environment.   So what’s next?  Remember our important CM word for CM studs only?  Yes, that’s right, it’s time for a Baseline…

Change Management: Starting with the basics

I was chatting in a crazed and maniacal manner with my coworker, Hobi, about my vision for change management for our team.  Hobi is a helpful guy to me, because he’s very good at forcing me to face the reality of my ignorance when necessary.  He threw me a few basic questions, to which I had to admit ignorance, and then, as I continued pontificating on my glorious plan I watched him slowly check out of the discussion.  He then stopped me in mid sentence and said, “Maybe you need to draw yourself a picture of what you’re trying to do.  See ya.”

Ok, so this is my cue to start with the basics.  The picture will come soon enough.  As my father (and little league baseball coach) always says, “Focus on the fundamentals and everything else will fall into place.”   Alright, so what is the hitting, throwing, catching, and base-running of change management?

In reading about Change Management, I quickly came across some other important related terms.

  • Service Request Management (Service Desk)
  • Incident Management
  • Problem Management
  • Release Management
  • Configuration Management

Holy Crap!!!  I hear these terms every day, and often hear them used interchangeably!  Oh no!  The only thing that is clear to me is that starting with the basics begins with understanding how to even title this blog!  I don’t know whether I’m trying to learn about Software CHANGE Management or Software CONFIGURATION Management.  Of course, as is so often the case in questions involving a choice, we can defer to the old slippery political side-step:  It’s really both, David.  It’s really both.

Fortunately, where there’s complexity in IT, there’s a standard.  The standard is ITIL:  Information Technology Infrastructure Library.  So, to help us understand how all of these processes work together, I defer to wikipedia .

Here’s your RAJP summary:

Service Request Management (Service Desk)

No brainer.  This is the user’s single point of contact for service regarding your software.  It covers everything, so I won’t bother listing everything.

Incident Management

Something screws up.  You fix it.

Problem Management

Something screws up.  You fix it.  It screws up again.  It’s a problem.  You investigate and fix the root problem that is causing the screw ups.

Change Management

The process you use to make changes to the system.  In the case of software, it’s called Software Change Management.  Commonly, SCM.

Release Management

Ok, so you’ve fixed some problems and you’ve added some new features.  At some point, your users are going to want to actually see these changes.  Release Management is the process of planning, communicating, building, QA, and deployment of a new release of your software.

Configuration Management

Your IT infrastructure is made up of a lot of components.   From the perspective of Configuration Management, every component is a Configuration Item (CI).   Configuration Management is all about tracking  changes on or between these items.  For SCM, your primary CI is your source code files.  So, we’re talking about baselines and source control.


Ok.  If you really want to talk the talk in SCM you have to use the word “baseline” about 250 times a day.  It’s such a simple sounding word that it can be used to intimidate the hell out the non-CM studs in your organization.  It sounds simple, so people are afraid to ask what it is (like me).  Well dammit!  I finally asked.   A baseline is the approved state of your software for starting changes for your next release.  This is probably going to be the state of your source code at the point of your last release.  So, an important goal of SCM is tracking the changes on every CI for a particular release from a specific BASELINE.  So if 3.1 was your last release, the source from 3.1 will serve as the baseline for your 3.2 release…simple.

Think this is all there is to ITIL?  Well, you’re wrong, buddy.  Dead wrong.  But this a good place to start.  So I’ll leave it at that.