BPM system cleanup options

Follow

INTRODUCTION:


There are several areas of BPM system level cleanup that should be done periodically to maintain a clean and performant system.  These operations should be done on a regular basis and should help you save disk space, and improve overall performance.
This documentation focuses on BPM 8.0 and above.


TOPICS:

  • Database structures and how they relate
  • Snapshot cleanup
  • Process instance cleanup
  • Performance Data Warehouse (PDW) cleanup
  • Durable Message Events
  • Conclusions


DATABASE STRUCTURES AND HOW THEY RELATE

As you may know, all BPM objects are stored in the BPM databases.  For the purposes in this article, we start by examining the following:

  • LSW_PROJECT – This is where all the process app(PA) and toolkit(TK) names are stored.  The columns we are most concerned with are:
    • NAME – the name of the PA/TK.  This is shown in the Process Designer and Process Center Console as the logical display name.
    • PROJECT_ID – the GUID of the PROJECT.  This will be used to lookup the various snapshots in the LSW_SNAPSHOT table.
  • LSW_SNAPSHOT – This is where all the named and unnamed snapshots will be listed.
    • PROJECT_ID – we use this to select the appropriate snapshots for a project we want to examine.
    • NAME – Name of the snapshot if a name has been specified (by a user creating a new snapshot), or null if the snapshot is unnamed.
  • LSW_DUR_MSG_RECEIVED – Where durable intermediate message events (IMEs) are stored.


SNAPSHOT/PROJECT CLEANUP


Before we talk about how to clean up the snapshot instances, we should understand why we need to do this.  The Process Center holds numerous snapshots of Process Apps (PA) and Toolkits (TK).  These are created in two ways as you develop them:

  • Named Snapshots – These are created whenever you take a deliberate snapshot from the Process Designer or Process Center Console.  The names snapshots can be deployed to Process Servers as well.
  • Unnamed Snapshots - An unnamed snapshot is created when you save in the Process Designer.  Unnamed snapshots only exist on the Process Center and are not included on exports.

NOTE: Archiving a project does not remove it from the database.  It simply marks the project in the database as archived and thus will not display in the UI.  This means we need to take extra steps to delete the projects.


NEW FUNCTIONALITY IN 8.5.0.1:


BPM 8.5.0.1 has introduced a new capability to enable automatic unnamed snapshot cleanup.  You can enable this by adding lines such as the following to your 100custom.xml file:

<unnamed-snapshots-cleanup-config>
    <enabled>true</enabled>
<cleanup-start-time>23:23:59</cleanup-start-time>
<cleanup-duration-minutes>5</cleanup-duration-minutes>
<clean-after-number-named-snapshots>4</clean-after-number-named-snapshots>
</unnamed-snapshots-cleanup-config>

See Deleting unnamed snapshots, automated in the IBM BPM V8.5 Information Center for more details.


OLDER VERSIONS OF BPM:
There are three paths to help with snapshot/project cleanup:

  • Remove a PA or TK:
    • Process Center
      • If you export (since the unnamed snapshots do not get exported) and then import into a NEW Process Center, then in the new Process Center, the project will have no unnamed snapshots.
      • Alternatively, you can delete projects (which will remove both snapshots, process instances, BPEL process instances, etc).  To do so:
        • First, assuming your PA/TK is not already archived:
          • Login to the Process Center Console
          • In the Process Apps tab (or Toolkits tab), select the PA/TK to archive
          • Click the “Manage” tab at the top
          • Click “Archive Process App”
          • When prompted, confirm you want to archive the PA.
        • Next, we need to delete the PA/TK
          • Login to the Process Center Console or Process Designer
          • Under the Process Apps tab (or Toolkits tab), select “Archived”
          • There you will see the PA/TK available.  Click the “Delete” button next to the desired PA/TK.
          • Confirm the operation by clicking “Delete” and you are done.
    • Process Server – online
      • Not available – cannot remove PA’s or TK’s.
    • Process Server – offline
      • Not available – cannot remove PA’s or TK’s.
  • Delete Snapshot
    • WARNING: Because issues have been discovered with the BPMDeleteSnapshot and BPMSnapshotCleanup commands, these commands require the interim fixes.  Please review those here: http://www-01.ibm.com/support/docview.wss?uid=swg21669992
    • Process Center
      • BPMSnapshotCleanup - Use the BPMSnapshotCleanup command to delete all the unnamed and archived snapshots of a process application on a Process Center server.
      • Pre-reqs:
        • Must be a repository administrator.
        • Cannot delete the first snapshot of a process app
        • You must archive named snapshots before deleting them.
        • To avoid conflicts, run the command when no operations are happening on the Process Center or any Process Designer client connected to it.
      • There are a number of ways to run the command.  Provided below are a few examples:
        • Delete snapshots between a time span:

wsadmin -conntype SOAP -port 4080 -host PC1.mycompany.com -user admin -password admin

$AdminTask BPMSnapshotCleanup {-containerAcronym BILLDISP
-createdAfterLocal 2012-01-01T00:00:00
-createdBeforeLocal 2014-02-31T21:37:06}

        • Delete unnamed snapshots using the createdBeforeLocal parameter

wsadmin -conntype SOAP -port 4080 -host PC1.mycompany.com -user admin -password admin
$AdminTask BPMSnapshotCleanup {-containerAcronym BILLDISP -createdAfterLocal 2012-05-31T21:38:00}

        • OR:

$AdminTask BPMSnapshotCleanup {-containerAcronym BILLDISP -createdAfterLocal 2012-07-31 -deleteArchivedSnapshot true

    • Process Server – online or offline
      • BPMDeleteSnapshot - Use the BPMDeleteSnapshot command in connected mode from Process Server to delete snapshots for process applications or toolkits.  You can only run the command for one process application or toolkit at a time. In a network deployment environment, you must run this command on the node containing the application cluster member that handles Process Server or Process Center applications. Do not run this command from the deployment manager profile.
      • Pre-reqs:
        • Must be a repository administrator
        • Snapshot must exist, be inactive, have no running instances, not be deployed.
        • On BPM Advanced, any business-level applications related to the snapshot must be uninstalled before you delete.
      • If the Pre-reqs are not met then in the SystemOut.log you will see:
        • You cannot delete an active snapshot.
      • Preparing for a snapshot deletion:
        • Run the BPMShowProcessApplication command to determine whether the snapshot exists for the process application.
        • Run the BPMShowSnapshot command to determine the status of the snapshot, such as whether it is the default snapshot and whether it is active with running instances.
        • Run the BPMDeactivate command to deactivate the snapshot.
        • Run the BPMStop command to stop the snapshot and its running instances.
        • Run the BPMUndeploy command to undeploy the snapshot from the server. This also uninstalls any business-level applications that are related to the snapshot.
        • NOTE: Using the SOAP connection to delete a snapshot can take longer to complete than the default SOAP timeout…this can result in an exception.  For more see: http://www-01.ibm.com/support/knowledgecenter/?lang=en#!/SSFPJS_8.0.1/com.ibm.wbpm.ref.doc/topics/rref_deletesnapshot.html
      • Example for BPMDeleteSnapshot:
        • The following examples shows how to establish a SOAP connection to Process Server and delete those snapshots for the process application BILLDISP that have snapshots acronyms of SS2.0.1 and SS2.0.2:

wsadmin -conntype SOAP -port 8880 -host ProcessServer01.mycompany.com -user admin -password admin


$AdminTask BPMDeleteSnapshot {-containerAcronym BILLDISP -containerSnapshotAcronyms {SS2.0.1 SS2.0.2}}

PROCESS INSTANCE CLEANUP

There are two main things to think about with for process instance cleanup:

  • Human services (tasks) associated with a process instance.
  • Process instances that are persisted even after the process completes (used for dashboards, reports and the like).  When deleted, the associated task instances are also deleted.
  • Task Cleanup Utility:
    • To access:
      • Login to the Process Admin Console
      • Expand IBM BPM Admin
      • Now you should see the Task Cleanup
      • Once you open the tool, choose the Cleanup Option you want and click “Cleanup”.
  • Process Instance Cleanup:
    • In BPM 8.0.1 and above, we can use the BPMProcessInstancesCleanup command: http://www-01.ibm.com/support/knowledgecenter/?lang=en#!/SSFPJS_8.0.1/com.ibm.wbpm.ref.doc/topics/rref_BPMProcessInstancesCleanup.html
    • Pre-reqs:
      • To run this command, you must be a member of the tw_admins group.
      • In a network deployment environment, use the port configured for the application cluster member that runs the Process Server or Process Center applications. To determine the correct port number, see the WebSphere administrative console Ports collection page (click Servers > Server Types > WebSphere application servers > server_name > Communications > Ports and find the value for SOAP_CONNECTOR_ADDRESS).
    • Examples:
      • Deleting instances that occur during a time range that falls within a specific range
        • wsadmin -conntype SOAP -port 8880 -host ProcessServer01.mycompany.com -user admin -password admin
        • $AdminTask BPMProcessInstancesCleanup {-containerAcronym BILLDISP -containerSnapshotAcronym SS2.0.1 -instanceStatus ALL -endedAfterLocal 2012-01-01T00:00:00 -endedBeforeLocal 2012-02-31T21:37:06}
      • Deleting instances that occur before the specified local time on the server
        • wsadmin -conntype SOAP -port 8880 -host ProcessServer01.mycompany.com -user admin -password admin
        • $AdminTask BPMProcessInstancesCleanup {-containerAcronym BILLDISP -containerSnapshotAcronym SS2.0.1 -instanceStatus ALL -endedBeforeLocal 2012-01-02T21:37:06}


PERFORMANCE DATA WAREHOUSE (PDW) CLEANUP

Firstly, there is no product support for purging PDW data at this time.


DURABLE MESSAGE EVENT CLEANUP

Some Intermediate Message Events (IME’s) are marked as durable.  This means they will not be removed once processed and thus over time, these events can accumulate and thus require some additional cleanup.

NEW FUNCTIONALITY:

This has been addressed in V7.5.1.2, V8.0.1.2, and V8.5.0.1 with a new wsadmin command named BPMDeleteDurableMessages, which takes three parameters:

  • olderThan: only events older than this number of days will be deleted
  • maximumDuration: the command will only run for this amount of time (in minutes)
  • transactionSlide: the number of events to delete per transaction

For example:

  • BPMDeleteDurableMessages {-olderThan 30 -maximumDuration 60 -transactionSlice 100}

OTHER BPM VERSIONS:

If you are not on the versions listed above, then there is no product supported way to remove durable message events, but if you want to check on how many there are, you can just issue this command in the BPM database:

  • select count (*) from LSW_DUR_MSG_RECEIVED

Removing any entries from the table above could have negative impacts on BPM as, for example, let's say a UCA concludes and it is used in an IME with durable checked.  A value is written to LSW_DUR_MSG_RECEIVED but if you remove that row, the association to the UCA is lost and the data (stored in the BLOB/CLOB) is invalid.

CONCLUSIONS

Overall, there are several areas of concern when looking at archiving and purging data.  Periodic reviews of your systems and cleanup within the BPM environment should help keep your systems running well.

Have more questions? Submit a request

Comments

  • Avatar
    David X Herring

    Hi Dave,
    Is there a utility available to make visible any business objects that are not being used in an application? or some way of detecting if there are dependencies between business objects?
    Thanks
    David Herring (Kaiser)?

  • Avatar
    Dave Rosen

    Hi David,

    Well there is no external activity I know of, but you an always do a "Where used" on a business object to identify which BPDs and Services are using is. This might also be in the scope of Neches, but I'll have to check on that (I simply don't recall at the moment).

  • Avatar
    Sagar Mankar

    Nice article Dave! Thanks for putting this together :)

  • Avatar
    Andres Felipe Gomez Mondragon

    Excellent article. Thanks for sharing these ones.

  • Avatar
    Dion Jones

    Hi Dave - brilliant insight, thanks
    When running an older version, do you have any suggestions for clearing LSW_DUR_MSG_RECEIVED any where near safely?

  • Avatar
    Dave Rosen

    Hard question Dion. To clean up previous UCA durable messages, use the BPMDeleteDurableMessages command (V8.5.0 Fix Pack 1 and later). Previous to that, there's no "safe" or supported way to do it, but take a look at this discussion...it should help: https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014833975#77777777-0000-0000-0000-000014833975

Powered by Zendesk