Integrating CA Software Change Manager with a Java EE application

CA’s Software Change Manager is a tool that we use to manage software and documents.  Although much of our development staff is using GIT, many of our engineers still use SCM (or Harvest as it used to be known).  The primary way our web applications use Harvest is as a version controlled repository for documents.  Our web apps can link directly to the latest versions of documents and can even allow our users to checkout and modify documents through the web.  I’d like to share some of the techniques we’ve used to build a relationship between a Java EE application apps and Harvest.

The obvious choice for using Harvest with java is the JHSDK (Java Harvest SDK).  But through trial and error, I’ve learned that JHSDK fails in Java EE because it’s classes are not thread safe.  At the CA World conference a few years ago, I had the privilege of consulting with the father of Harvest (can’t remember his name) about this problem.  He said that I have two options, make the API thread safe or just make system calls to the command line interface (Harvest CLI).  Not knowing exactly how to make the API thread safe, I chose the latter.  It has never failed us.

Things to consider when designing a Harvest-related web app:

1.  The server on which your app server resides must have a Harvest client installed.   You’re essentially creating an automated Harvest user in your web app.

2.  Because you’re using a CLI instead of JHSDK, you have to retrieve errors and exceptions by reading logs.  Each Harvest command creates its own log file on the server.  So you have to manage log files in real time.  We create a new log file with a new random name with every command.  After the command runs, we check the log messages so that we can send the messages (including errors) back to the browser.  And finally, we delete the log file.  This is handled differently for asynchronous calls (see #5).

3.  Each Harvest command must contain user credentials.  When the user logs in, you could capture his/her username and password, but this isn’t very secure.  Ultimately, you want to use an auth file on the server.  This can be generated with a command using the username and password one time and never again (unless the password changes).  You name the auth file after the user name and then you can reference it anytime you need it.  The svrenc command looks like this:

String[] command = new String[11];
int i = 0;
command[i++] = “svrenc”;
command[i++] = “-f”;
command[i++] = userName+”.auth”;
command[i++] = “-usr”;
command[i++] = userName;
command[i++] = “-pw”;
command[i++] = password;
command[i++] = “-dir”;
command[i++] = siService.getAuthRootPath();
command[i++] = “-o”;
command[i++] = log;

4.  Building commands can be error prone if it’s done in one big String.  Fortunately, Java’s Runtime class takes a String array.  It’s best to build your commands this way. (see above example for building a command)

5.  Commands can be run asynchronously for long-running processes by putting the command into a thread.  As the thread runs it writes the progress of the log file output into the database and the client polls it with AJAX calls.  That way you can show progress on the process.

6.  When the user needs to view a file, it’s easier and quicker to use SQL rather than the hco command to get a blob and stream it out to the browser

7.  When the user needs to checkout a file (hci command), you’re checking it out to the server’s file system then streaming a copy of that file back to the web browser.

8.  To check in a file, upload the file through the web browser to the exact spot where it was checked out to, then run the hco command on it.

9.  Finally, use the documentation.  It comes with your installation and is called “CA Software Change Manager Command Line Reference Guide”.  Everything you can do with the SCM client can be done via the CLI, and therefore can be done in a web app.

Integrating your web apps with CA SCM can be a very powerful asset to your users.  We allow users to list, view, and manage files, promote/demote packages, edit forms, create comments, and approve/deny packages.  We had hoped that CA would be a decent web version of SCM, but it never happened, so we built the parts that we need.  We’ve been very successful, and using CLI calls has been very reliable.

Advertisements

Netbeans: Session Beans for Entity Classes

Netbeans really hit a home run when it automated the creation of session beans from entity classes.  It also does a good job creating entity beans.  It’s a win-win (God!  I can’t believe I just wrote the phrase win-win).   It makes good use of the facade design pattern and gives you a service that you can inject into your controller.

Generated Entity Beans

First, let’s talk about what you get when you let Netbeans generate your entity beans.  You might say, “Well Mr. RAJP, I’m perfectly capable of creating my own damn entity beans.”  To which I say “Fiddlesticks.  You’re wasting your employer’s valuable time!”

The entity bean that Netbeans generates gives you everything you need (and perhaps more) just by selecting a table in a list.  Netbeans generated entity bean come with the following features:

  1. Declares each field in the table with an estimation of the proper type and annotates maximum sizes, default values,  whether it can take a null value, and whether it is the primary key.
  2. It maps all of the database relationships with @OneToMany, @ManyToOne, and @ManyToMany depending on how well you designed your database.
  3. It creates a nested array of @NamesQueries searching on each field.
  4. It generates all the getters and setters
  5. It overrides hashCode() , toString(), and equals()

The only modifications I ever have to make is to add a sequence generator to the primary key and add a few more named queries.

“But David! This is too easy.  Shouldn’t I have to work harder to get my job done?”

“No.  This gives you more time to build your damn system.  Now swallow your pride and code!”

Generated Session Beans for Entity Beans

This is where things get really cool.  For years I’ve been coding a service class, and doing it poorly.  Being a lowly RAJP (Regular Average Java Programmer), I was building a static service of operations to perform on the entity beans.  Basic CRUD plus a few custom business functions.  Did it work?  Yes.  Should I have created some sort of service that could be injected into my controller?  Yes.  Am I too lazy to do anything that cool?  Definitely.

When I upgraded to Netbeans 7.1, and was generating some entity beans, I happened to notice an option to generate “Session Beans for Entity Classes…” in the New menu.  Honestly, I had no idea what this would do.  When I did it though, which was as simple as selecting from a list of entity beans, it gave me something totally cool.  It gave me a set of stateless EJB classes replete with EntityManagers already hooked up to my persistence unit.  But wait!  There’s more!  It gave me a nicely designed AbstractFacade for the EJBs to extend with all the CRUD methods I would need.  It’s so nicely designed, in fact, that it can be extended by any session bean you generated from an entity.  It’s all generical and stuff.

Netbeans got it right!  This is a far better a design than what I could have produced myself.  This is a design worthy of a JE (Java Expert).

The only slightly weird thing is that the session bean never uses the named queries in the entity bean.  It handles all of it’s operations with JPAs CriteriaQuery class.  But that’s just fine with me.  When I need it, I just add another operation to the session bean to call named queries.

Now I have a set EJB facades that I can inject into my controllers.  I use Stripes ActionBean for my controllers, by the way.  It’s a kick ass stack.  JSP, Stripes, EJBs, JPA, Glassfish, and Netbeans.

Netbeans team, you NAILED this down TIGHT! Yessir!

Simple Authentication Solution with the Stripes Framework

This security solution will probably not stand up to the scrutiny of a JE (Java Expert), but I’m cool with it.  It’s probably all you’re really looking for.  Something basic and simple.  If you’re new to Stripes check out Stripes Quick Start Guide

The solution requires 4 elements:

  1. @DoesNotRequireLogin Annotation
  2. Security Interceptor
  3. Security Action Bean
  4. JSP login form

@DoesNotRequireLogin Annotation

Creating a Java annotation is very simple.  As a RAJP (Regular Average Java Programmer), I was intimidated at first.  What if I’m not smart enough to figure this out?  What will the JEs think?  ARRRG!  Relax.  It’s easy

That’s it.  You have an annotation.  Now let’s see what you can do with it!

Security Interceptor

see Stripes’ documentation on interceptors http://www.stripesframework.org/display/stripes/Intercept+Execution

An interceptor breaks in at a certain stage of the Stripes life cycle.  In this case, we want it to run any time an action is requested so that we can see if the action requires a login.  Then we make a decision.  Either let the request pass through, or do something else.  In this case we’re asking if the Action Bean that is being requested is annotated with @DoesNotRequireLogin.  I set it up that way instead of @RequiresLogin because the only thing that doesn’t require login is the login.  So I annotated that bean with @DoesNotRequireLogin.  Any other bean has to pass the treacherous isLoggedIn method.  Anything that fails this gets returned back to the login screen.  Also notice that we are implementing the Interceptor interface of which the intercept method is required for implementation.

Action Bean

The action bean serves as the controller.  It can be called upon by any page by setting the beanClass property of a form or link.  The @UrlBinding annotation is where you specify why URL will map to this bean.  The default is to go to the login event because it’s annotated with @DefaultResolution.  All this does is forward the request to the login jsp.  The first instance variable is an EJB that handles data access for the login transaction.  The next two are the variables that will be bound to the login form.  The submitLogin method is the event that the login form will call.  Got it?  That was a lot.  You may need to reread and take a closer look at the tiny picture of my code.

Notice that the class has a @DoesNotRequireLogin annotation.  A request to this Action Bean will pass through the security interceptor.  The submitLogin event processes the login form and uses the EJB to decide whether it passes.  If it doesn’t, we add an error message and return the request back to the login screen.  If it passes, it uses the path variable to forward the request on to wherever the user was trying to go in the first place.

Notice the annotations on the userId and password.  This is Stripes super easy way of doing form validations.  It runs the validation before the action event (submitLogin)  is run and will automatically return to your form with errors if they fail.

Finally, you need a logout event.  This returns the loggedIn session attribute back to false and forwards the request back to the login form.  You’ll just need to throw a logout link somewhere in your application.

Login Form

The login page is a very basic login using a handful of Stripes tags.

It’s a little clipped so you don’t see the layout rendering and the closing div and such, but that’s not what’s important here.   I want you to note just a few things.

  1. The form uses Stripes tags instead of regular HTML tags.  You tell it what Action Bean to submit the form to (see beanclass in the stripes:form…this is our security action bean), and then you tell it what event should handle it (see the stripes:submit tag name property…this is the submitLogin event in the security action bean)
  2. When you use Stripes tags for the inputs, Stripes automagically maps the data input to the instance variables in the Action Bean .  The values of the name attributes have to match the variable names in the action bean (userId and password which have to have getters and setters for it to work).  Pretty handy if you’re used to getting stuff out of the Request and setting them into the variables yourself.
  3. Look at the <stripes:errors> tag.  If the security action bean says the login failed or if the input fields don’t pass validation, it forwards the request back to the login screen and displays the errors in the stripes:errors tag.  Simple. Elegant.
That’s it.  Any request that comes into this application will be intercepted by SecurityInterceptor, and if the requested action bean does not have the @DoesNotRequireLogin, it will ask if the person is logged in, etc.  And in our case, the security action bean will be accessed.  It will route the request to the login page.  The form will submit back to the security action bean (and will pass the interceptor) and go straight to the submitLogin event for processing.  If the login should fail, the request will be returned to the login page with an error message that says so; otherwise, it’s on to the application!
Ok, JEs (Java Experts), you’re welcome to cut this into shreds, but the rest of RAJPs are satisfied…at least this one is.

Please Use the Stripes Framework

I’ve use Struts and played with Spring.   Stripes is so much easier.  It takes Java EE and makes it almost as fun as using something like Ruby on Rails.  It is annotations over configuration framework, which may take a little getting used to, but once you’re into it you just sail right along into programming heaven.

I’m just a Regular Average Java Programmer (RAJP), so I can’t give you a technical point by point on why to use Stripes.  I’ll leave that to the experts.  Stripes:  A successful first project

But here’s the way I look at it.  Stripes is focused on one thing:  request plumbing.

It takes a GET or a POST and sends it to the right Action Bean (controller) and to the right event (Action Bean method).  The Action Bean does stuff with the data and then chooses a view.    It maps data from the model to the view and it maps data from the view into the model.  And then it give you really cool little helpers like validation, layout managers, and interceptors to handle cross-cutting concerns like, say, security.

It does not persist data.  It says, “Hey, there’s lot’s of cool ways to handle data.  That’s not our job.  Do it how ever you want to.”  (see EJB and JPA and DAO/Transfer Object, Stored Procedures, and such).

It does not (like JSF) give you fancy view components for building web pages.  It says ,”Hey, there’s all kinds of cool stuff that web developers use like HTML, JSP, CSS, and Javascript.  Have fun!”

It does not handle dependency injection.  It says “Hey! There’s these things called EJBs and Spring Beans that do DI really well.  Give them a try!”

It’s light-weight and focused on wiring things together without a bunch of XML.  It’s goal is to handle all of the wiring for data binding and navigation (much like Ruby on Rails) in order to free you up to build your system.

I use it with JSP, JPA and EJBs.  One guy on my team uses Spring Beans instead of EJBs.  One gal on my team uses JDBC DAO and Stored procedures instead of JPA.  If you like, you could use a different view framework like FreeMarker.  Stripes doesn’t care.  It plays well with others.

The more I use it, the better I like it.  It has never disappointed!

Please use Stripes.  We’re a small, but very satisfied community.  More people should be using it.

Deploying JHSDK in a Java EE environment

[EDIT:  Ok. New info. JHSDK is not thread-safe. We’ve been very successful wrapping the CDI commands in java classes.  JHSDK is only appropriate for a Java SE environment.  Integrating CA Software Change Manager with a Java EE application]

Introduction

Deploying a web application that uses the JHSDK is not as simple as throwing a jar file in your lib directory. The JHSDK is a little different than other java libraries on three counts:

  1. it is a java wrapper for a C++ library. Allfusion Software Change Manager 7.x is a C++ application. The upcoming v12 release is a java application, so this C++ layer will go away [EDIT:  They did not update the JHSDK  to be pure java.  This prediction was incorrect]. But for now, we’re stuck with it.
  2. it is a bridge for communicating with another enterprise system. Harvest is a non-web, n-tier system for managing change on software and documents. It has a proprietary front controller that brokers requests from various Harvest clients.
    Harvest Architecture from \" width=
  3. it’s classes are not serializable, which presents problems when using it in Java EE environment

So, for your Java EE application to use JHSDK it must

  1. know where the C++ libraries reside
  2. run in an environment that has a Harvest client installed
  3. use serializable wrappers for the JHSDK classes that you need to persist

CA provides detailed documentation for setting up the environment in their Installation Guide. If you’re not accustomed to installing a Harvest client, this can be somewhat challenging. I found it very easy to deploy my web application to my local windows installation of Glassfish because I already have a working Harvest client installed; however, I worked for nearly three weeks with my Sun admin, Harvest admin, and CA Support to configure our development Solaris environment. We had never used a Harvest client on Solaris before, and you know how UNIX environments are: SOME ASSEMBLY REQUIRED.

Windows Deployment

This is an account of my deployment to a Java EE platform on Windows

OS: Windows XP SP2

Application Server: Sun Java System Application Server 9.0 Update 1 (glassfish v2 ur1)

Harvest: AllFusion Harvest Change Manager 7.0

I didn’t have to set up the Harvest Client because it was already done, so I won’t comment on that.

In Windows, there are two files that Glassfish needs to know about: jhsdk.jar and JHSDK.dll. Both of these files need to be your Glassfish’s path.

You can put them in

GLASSFISH_HOME\lib

or

GLASSFISH_HOME\domains\domain1\lib

If you deploy the jhsdk.jar file without the JHSDK.dll file, then you will receive the following error:

java.lang.UnsatisfiedLinkError: no JHSDK in java.library.path

At this point in the development process, my application provides a list of versions for a particular Harvest Package. The JCaVersion class does not implement java.io.Serialiable, so I was not able to bind it to my JSF component. So I wrote my own version of JCaVersion. I looped through my array of JCaVersions (which is returned from JCaPackage getVersionList()), and built an array of MyVersion instances with the same data. Then I could bind my array of MyVersion instances to my JSF component.

Finally, I deployed my WAR file and I was ready to go. I was so excited. WOW! That was easy! I’ll just deploy it on the Solaris box and I’ll be ready for show and tell. And……PRESTO!!! 3 weeks, 5 support emails to CA Support, a chat fight with my Sun admin, 50 meetings with my Harvest Admin later, it’s working!

Solaris Deployment

OS: Solaris 9

Application Server: Sun Java System Application Server 9.0 Update 1 (glassfish v2 ur1)

Harvest: AllFusion Harvest Change Manager 7.0

The first issue that arose was that the client was not fully installed on the Solaris box. Our app server and the Harvest server happen to be installed on the same machine, so the client should have been installed and available already. The key command-line utilities were there, and the HSDK and JHSDK are supposed to come with the command-line utilities, but they didn’t. So, we reinstalled the command-line utilities, and that set up all the files and directories we were needing.

Once we had the client installed, we needed to configure an environment for using the JHSDK. CA’s installation guide is pretty explicit, but I had trouble using it…perhaps because I’m not really that skilled with Solaris. Here’s what my env looks like:

JAVA_HOME=/usr/java
HARVESTHOME=/xxx/ca/harvest
CACRYPTINI=/opt/CA/CAcrypto/cacrypt.ini
CACRYPTDIR=/opt/CA/CAcrypto
DVHOME=/xxx/ca/pec/graphics/sun4_solaris
RTARCH=sun4_solaris
RTARCH_UC=SUN4_SOLARIS
RTHOME=/xxx/ca/pec
_app=/xxx/ca/pec/app-defaults
export JAVA_HOME HARVESTHOME CACRYPTDIR CACRYPTINI
export DVHOME RTARCH RTARCH_UC RTHOME _app
LD_LIBRARY_PATH=${CACRYPTDIR}:${RTHOME}/lib/sun4_solaris:${RTHOME}/graphics/sun4_solaris/lib:/usr/lib
LD_LIBRARY_PATH=${HARVESTHOME}/lib:${HARVESTHOME}/HSDK/lib:${HARVESTHOME}/JHSDK/lib:$LD_LIBRARY_PATH
PATH=${RTHOME}/bin:${HARVESTHOME}/bin:$PATH
export LD_LIBRARY_PATH PATH

I recommend following CA’s advice to test your environment with the sample JHSDK application as detailed in the Installation Guide. This will tell you whether or not you were successful.

The next step is to configure Glassfish. This is where it gets weird. Even though we’ve setup all this path info in the environment, we have to do it again in Glassfish. My guess as to why? The Java EE environment needs to know the paths to the native libs so the when the JHSDK classes are called, then can know where to find their C++ peers. The user environment needs to know the paths so that when when one of the native C++ classes is called, it knows where they are and where Harvest is. Again, that’s my best guess. If I learn different, or if you know different please tell me and I will give an update.

So here’s what my Native Library Path Suffix in Glassfish looks like:

:/xxx/ca/harvest/lib:/xxx/ca/harvest/HSDK/lib:/xxx/ca/harvest/JHSDK/lib:/opt/CA/CAcrypto:/xxx/ca/pec/lib/sun4_solaris:/xxx/ca/pec/graphics/sun4_solaris/lib:/usr/lib

Here’s what my Classpath Suffix looks like:

:/usr/java/jre/lib:/usr/java/lib:/xxx/ca/harvest/lib:/xxx/ca/harvest/HSDK/lib:/xxx/ca/harvest/JHSDK/lib:/xxx/ca/harvest/JHSDK/lib/jhsdk.jar

Look for future posts here on my use of JHSDK in my Java EE application. The journey has only begun…

Integrating a Java EE application with CA Allfusion Software Change Manager

Our organization uses a Computer Associates product called CA Software Change Manager to manage change on code and documentation.  It used to be called Harvest (much easy to say), so please excuse me, CA, if I go ahead and refer to it as Harvest from this point forward.  It’s a very robust and very extensible CM platform.   In the past, I’ve built custom extensions for Harvest for our division using SQL and a set of command-line utilities that ship with the product.   I’ve been aware for some time, however, that CA also ships the Java Harvest SDK.  This is supposed to be the best way to extend and integrate with Harvest.

Here’s a picture of my proposed architecture.  Notice where the JHSDK fits in with it.

fig17.gif

First major issue I will have to address:

Harvest connection management.  JHSDK has a class called com.ca.harvest.jhsdk.JCaHarvest that represents an active session with the Harvest application.  My plan is to pool some instances of these classes with a generic webuser account for browse/read-only stuff, but make them authenticate when they need to checkin a new file or checkout a file for update (thereby creating new JCaHarvest instances with their specific logins).

This is an exciting bit of development for me!   This wouldn’t be a bad topic for a CA World conference.  First things first, let’s get this thing working!

[EDIT:  JHSDK blows for a web app.  You have to create a service class that wraps system calls to the Harvest CDI…which works just fine]