Integrating CA Software Change Manager with a Java EE application

CA’s Software Change Manager is a tool that we use to manage software and documents.  Although much of our development staff is using GIT, many of our engineers still use SCM (or Harvest as it used to be known).  The primary way our web applications use Harvest is as a version controlled repository for documents.  Our web apps can link directly to the latest versions of documents and can even allow our users to checkout and modify documents through the web.  I’d like to share some of the techniques we’ve used to build a relationship between a Java EE application apps and Harvest.

The obvious choice for using Harvest with java is the JHSDK (Java Harvest SDK).  But through trial and error, I’ve learned that JHSDK fails in Java EE because it’s classes are not thread safe.  At the CA World conference a few years ago, I had the privilege of consulting with the father of Harvest (can’t remember his name) about this problem.  He said that I have two options, make the API thread safe or just make system calls to the command line interface (Harvest CLI).  Not knowing exactly how to make the API thread safe, I chose the latter.  It has never failed us.

Things to consider when designing a Harvest-related web app:

1.  The server on which your app server resides must have a Harvest client installed.   You’re essentially creating an automated Harvest user in your web app.

2.  Because you’re using a CLI instead of JHSDK, you have to retrieve errors and exceptions by reading logs.  Each Harvest command creates its own log file on the server.  So you have to manage log files in real time.  We create a new log file with a new random name with every command.  After the command runs, we check the log messages so that we can send the messages (including errors) back to the browser.  And finally, we delete the log file.  This is handled differently for asynchronous calls (see #5).

3.  Each Harvest command must contain user credentials.  When the user logs in, you could capture his/her username and password, but this isn’t very secure.  Ultimately, you want to use an auth file on the server.  This can be generated with a command using the username and password one time and never again (unless the password changes).  You name the auth file after the user name and then you can reference it anytime you need it.  The svrenc command looks like this:

String[] command = new String[11];
int i = 0;
command[i++] = “svrenc”;
command[i++] = “-f”;
command[i++] = userName+”.auth”;
command[i++] = “-usr”;
command[i++] = userName;
command[i++] = “-pw”;
command[i++] = password;
command[i++] = “-dir”;
command[i++] = siService.getAuthRootPath();
command[i++] = “-o”;
command[i++] = log;

4.  Building commands can be error prone if it’s done in one big String.  Fortunately, Java’s Runtime class takes a String array.  It’s best to build your commands this way. (see above example for building a command)

5.  Commands can be run asynchronously for long-running processes by putting the command into a thread.  As the thread runs it writes the progress of the log file output into the database and the client polls it with AJAX calls.  That way you can show progress on the process.

6.  When the user needs to view a file, it’s easier and quicker to use SQL rather than the hco command to get a blob and stream it out to the browser

7.  When the user needs to checkout a file (hci command), you’re checking it out to the server’s file system then streaming a copy of that file back to the web browser.

8.  To check in a file, upload the file through the web browser to the exact spot where it was checked out to, then run the hco command on it.

9.  Finally, use the documentation.  It comes with your installation and is called “CA Software Change Manager Command Line Reference Guide”.  Everything you can do with the SCM client can be done via the CLI, and therefore can be done in a web app.

Integrating your web apps with CA SCM can be a very powerful asset to your users.  We allow users to list, view, and manage files, promote/demote packages, edit forms, create comments, and approve/deny packages.  We had hoped that CA would be a decent web version of SCM, but it never happened, so we built the parts that we need.  We’ve been very successful, and using CLI calls has been very reliable.

Advertisements

Step 2: Set up a Repository

Before you can load a baseline for your release, you have to set up a place to put it.   In CA SCM, this is called a Repository.   The definition of Repository in the documentation is very simple:

A repository is a collection of items and item paths

You can think of Items as files and item paths as folders.  In truth, an item is  the information about a single Configuration Item.  The file itself, is always a Version of an Item.   So a repository is a collection of Items and their Versions organized with Item Paths.

An administrator can load files into a repository with the admin tool, but the only way a regular user can affect a repository is through Packages in a Life Cycle. What’s really cool is that a Repository can be associated with one or more Life Cycles.   So you might have different kinds of processes that manage change on the same configuration items.   This way, you don’t have to create one MONSTER process to manage everything.  You can design Life Cycles that manage your various processes from Service Desk, to Release Management, to Maintenance, etc.

Steps to Setting up a Repository:

  1. Login to CA SCM Administrator and select the Repositories tab

    CA SCM Administator Repository Tab

    CA SCM Administator Repository Tab

  2. Right-click the broker node and select New Repository

    Right-Click Broker node

    Right-Click Broker node

  3. Give your Repository a unique name.   Ok, this is a good time talk about a choice you need to make.  Will your repository be used over and over again for many iterations and have a long trail of versions?  Or will it retain only the development iterations on a single release?  In my case, I’m focused on the development of a single release, so my repository name will be the name of my release.

    Repository Properties Dialog

    Repository Properties Dialog

  4. Click Apply
  5. Setup Access to your repository.  Click the Access tab of the Repository Dialog.   Note that there are three Access Types in the drop-down field:  Secure Access, Update Access, and View Access.  I’m not sure what Secure Access is.  I just set it to the Administrator group.  Update Access needs to go to your developers.  View Access needs to go to your developers and any other group that needs to view the code.
  6. Click Ok.  Congratulations, you have a place to put your crap.
  7. Load your Baseline into the repository.   If you’re using Eclipse and the CA SCM Plugin, it’s important that you load each entire Eclipse project that you are using to work with your software.  Right-click your Repository node and select Load Repository.  This will bring up the Load Repository Dialog
    Load Repository Dialog

    Load Repository Dialog

    Browse out to your local machine to find the files that will make your baseline.  Check Recursive so that you can get all the files and folders in you projects.  Use “*” in the Files field to get all file types.  If you set up some subfolders in your Repository for this project or that, your can specify them in your Repository Path field.

    NOTE:  You can create a repository by duplicating another repository, so the files from your last release are already loaded.

  8. Click Load and watch it go!

CRUCIAL STEP

Just because you have a place to put your crap, doesn’t mean you have access to it from your Life Cycle.   The key to CA SCM is Association.  This is what gives the tool it’s flexibility.  You must add your repository to the Baseline of your Life Cycle.

  1. Navigate to the Baseline node of your Life Cycle

    Baseline Node

    Baseline Node

  2. Right-click node and select Configure Baseline
    fig35
  3. Add one or more Repositories from the Available Repository/View List

    Configure Baseline Dialog

    Configure Baseline Dialog

Woo Hoo!   Now, this is where it gets fun!

Deploying JHSDK in a Java EE environment

[EDIT:  Ok. New info. JHSDK is not thread-safe. We’ve been very successful wrapping the CDI commands in java classes.  JHSDK is only appropriate for a Java SE environment.  Integrating CA Software Change Manager with a Java EE application]

Introduction

Deploying a web application that uses the JHSDK is not as simple as throwing a jar file in your lib directory. The JHSDK is a little different than other java libraries on three counts:

  1. it is a java wrapper for a C++ library. Allfusion Software Change Manager 7.x is a C++ application. The upcoming v12 release is a java application, so this C++ layer will go away [EDIT:  They did not update the JHSDK  to be pure java.  This prediction was incorrect]. But for now, we’re stuck with it.
  2. it is a bridge for communicating with another enterprise system. Harvest is a non-web, n-tier system for managing change on software and documents. It has a proprietary front controller that brokers requests from various Harvest clients.
    Harvest Architecture from \" width=
  3. it’s classes are not serializable, which presents problems when using it in Java EE environment

So, for your Java EE application to use JHSDK it must

  1. know where the C++ libraries reside
  2. run in an environment that has a Harvest client installed
  3. use serializable wrappers for the JHSDK classes that you need to persist

CA provides detailed documentation for setting up the environment in their Installation Guide. If you’re not accustomed to installing a Harvest client, this can be somewhat challenging. I found it very easy to deploy my web application to my local windows installation of Glassfish because I already have a working Harvest client installed; however, I worked for nearly three weeks with my Sun admin, Harvest admin, and CA Support to configure our development Solaris environment. We had never used a Harvest client on Solaris before, and you know how UNIX environments are: SOME ASSEMBLY REQUIRED.

Windows Deployment

This is an account of my deployment to a Java EE platform on Windows

OS: Windows XP SP2

Application Server: Sun Java System Application Server 9.0 Update 1 (glassfish v2 ur1)

Harvest: AllFusion Harvest Change Manager 7.0

I didn’t have to set up the Harvest Client because it was already done, so I won’t comment on that.

In Windows, there are two files that Glassfish needs to know about: jhsdk.jar and JHSDK.dll. Both of these files need to be your Glassfish’s path.

You can put them in

GLASSFISH_HOME\lib

or

GLASSFISH_HOME\domains\domain1\lib

If you deploy the jhsdk.jar file without the JHSDK.dll file, then you will receive the following error:

java.lang.UnsatisfiedLinkError: no JHSDK in java.library.path

At this point in the development process, my application provides a list of versions for a particular Harvest Package. The JCaVersion class does not implement java.io.Serialiable, so I was not able to bind it to my JSF component. So I wrote my own version of JCaVersion. I looped through my array of JCaVersions (which is returned from JCaPackage getVersionList()), and built an array of MyVersion instances with the same data. Then I could bind my array of MyVersion instances to my JSF component.

Finally, I deployed my WAR file and I was ready to go. I was so excited. WOW! That was easy! I’ll just deploy it on the Solaris box and I’ll be ready for show and tell. And……PRESTO!!! 3 weeks, 5 support emails to CA Support, a chat fight with my Sun admin, 50 meetings with my Harvest Admin later, it’s working!

Solaris Deployment

OS: Solaris 9

Application Server: Sun Java System Application Server 9.0 Update 1 (glassfish v2 ur1)

Harvest: AllFusion Harvest Change Manager 7.0

The first issue that arose was that the client was not fully installed on the Solaris box. Our app server and the Harvest server happen to be installed on the same machine, so the client should have been installed and available already. The key command-line utilities were there, and the HSDK and JHSDK are supposed to come with the command-line utilities, but they didn’t. So, we reinstalled the command-line utilities, and that set up all the files and directories we were needing.

Once we had the client installed, we needed to configure an environment for using the JHSDK. CA’s installation guide is pretty explicit, but I had trouble using it…perhaps because I’m not really that skilled with Solaris. Here’s what my env looks like:

JAVA_HOME=/usr/java
HARVESTHOME=/xxx/ca/harvest
CACRYPTINI=/opt/CA/CAcrypto/cacrypt.ini
CACRYPTDIR=/opt/CA/CAcrypto
DVHOME=/xxx/ca/pec/graphics/sun4_solaris
RTARCH=sun4_solaris
RTARCH_UC=SUN4_SOLARIS
RTHOME=/xxx/ca/pec
_app=/xxx/ca/pec/app-defaults
export JAVA_HOME HARVESTHOME CACRYPTDIR CACRYPTINI
export DVHOME RTARCH RTARCH_UC RTHOME _app
LD_LIBRARY_PATH=${CACRYPTDIR}:${RTHOME}/lib/sun4_solaris:${RTHOME}/graphics/sun4_solaris/lib:/usr/lib
LD_LIBRARY_PATH=${HARVESTHOME}/lib:${HARVESTHOME}/HSDK/lib:${HARVESTHOME}/JHSDK/lib:$LD_LIBRARY_PATH
PATH=${RTHOME}/bin:${HARVESTHOME}/bin:$PATH
export LD_LIBRARY_PATH PATH

I recommend following CA’s advice to test your environment with the sample JHSDK application as detailed in the Installation Guide. This will tell you whether or not you were successful.

The next step is to configure Glassfish. This is where it gets weird. Even though we’ve setup all this path info in the environment, we have to do it again in Glassfish. My guess as to why? The Java EE environment needs to know the paths to the native libs so the when the JHSDK classes are called, then can know where to find their C++ peers. The user environment needs to know the paths so that when when one of the native C++ classes is called, it knows where they are and where Harvest is. Again, that’s my best guess. If I learn different, or if you know different please tell me and I will give an update.

So here’s what my Native Library Path Suffix in Glassfish looks like:

:/xxx/ca/harvest/lib:/xxx/ca/harvest/HSDK/lib:/xxx/ca/harvest/JHSDK/lib:/opt/CA/CAcrypto:/xxx/ca/pec/lib/sun4_solaris:/xxx/ca/pec/graphics/sun4_solaris/lib:/usr/lib

Here’s what my Classpath Suffix looks like:

:/usr/java/jre/lib:/usr/java/lib:/xxx/ca/harvest/lib:/xxx/ca/harvest/HSDK/lib:/xxx/ca/harvest/JHSDK/lib:/xxx/ca/harvest/JHSDK/lib/jhsdk.jar

Look for future posts here on my use of JHSDK in my Java EE application. The journey has only begun…

Integrating a Java EE application with CA Allfusion Software Change Manager

Our organization uses a Computer Associates product called CA Software Change Manager to manage change on code and documentation.  It used to be called Harvest (much easy to say), so please excuse me, CA, if I go ahead and refer to it as Harvest from this point forward.  It’s a very robust and very extensible CM platform.   In the past, I’ve built custom extensions for Harvest for our division using SQL and a set of command-line utilities that ship with the product.   I’ve been aware for some time, however, that CA also ships the Java Harvest SDK.  This is supposed to be the best way to extend and integrate with Harvest.

Here’s a picture of my proposed architecture.  Notice where the JHSDK fits in with it.

fig17.gif

First major issue I will have to address:

Harvest connection management.  JHSDK has a class called com.ca.harvest.jhsdk.JCaHarvest that represents an active session with the Harvest application.  My plan is to pool some instances of these classes with a generic webuser account for browse/read-only stuff, but make them authenticate when they need to checkin a new file or checkout a file for update (thereby creating new JCaHarvest instances with their specific logins).

This is an exciting bit of development for me!   This wouldn’t be a bad topic for a CA World conference.  First things first, let’s get this thing working!

[EDIT:  JHSDK blows for a web app.  You have to create a service class that wraps system calls to the Harvest CDI…which works just fine]