When we talk about IBM BPM web services (WS) we typically refer to two types - inbound or outbound web services.
Inbound web services (WS) in IBM BPM is when you create a web service in BPM, it's hosted by BPM and you use an external call to BPM calling that web service to trigger a specific action in your process.
Outbound web service (WS) IBM BPM is when you're making a call to an external web service (WS) that is hosted outside of IBM BPM.
Web services (WS) stack in IBM BPM is implemented by underlying WebSphere JAX-WS web service engine. It's possible to fall-back to a previous web service engine that was used in earlier IBM BPM (Lombardi edition) releases but this is outside of the scope of this article. So, we will assume that JAX-WS is used which is the default web service engine in IBM BPM 8.5.x.
Examples of web services performance issues
Performance depends on too many factors. You might have a situation where you make a number of WS calls within the BPM and these calls average around 0.5 seconds (500 ms) per call, when executed through BPM, but have response times 5-10 faster if called outside of the BPM context (SOAP UI as an example).
One of the common reasons why IBM BPM SOAP connectors (actually pretty much any JAX-WS implementation) are going to be slower than HTTP GET and soapUI is that the SOAP connectors parse and validate the WSDL and XSDs as well as serialize/deserialize the SOAP Request/Response to/from BPM engine variables. All these operations have a cost (cost of simplicity, reliability and speed of development). Therefore, all these aspects need to be taken into account. In other words, comparing performance HTTP GET to Web Service Integration is NOT "apples to apples" comparison.
Another aspect to keep in mind is the caching of parsed WSDL and XSDs on the BPM side (for outbound web services), which means that the FIRST invocation of the web service after JVM restart or WSDL/XSD update is always going to be slower than subsequent invocations (because of all the parsing of WSDL/XSD BPM has to do). Therefore, when you test for performance, it is better to make sure you execute the test multiple times and measure the moving average.
It's also important to keep in mind that most operations related to WSDL and XSDs parsing is CPU intensive operations. So, you might be getting significantly different results on different servers with different HW configuration(s).
How to troubleshoot web services performance issues
- It's best to narrow down the problem to something specific when using WS integration or plain SOAP connector. In order to do that you can use generic WS traces from the following technote or use so called WSPerf trace documented in the following article. The advantage of WSPerf trace is that it's lightweight, when compared to the full web services engine trace.
- Make sure to monitor OS resources during your WS performance tests. Monitor from an OS level (CPU, memory) as well as from a WAS/JVM level (memory/heap, verboseGC, threads). If your IBM BPM server is running in a Virtual Machine make sure you analyze CPU consumption from a Virtual Machine ESX server perspective because if you do have shared resources in your VM appliance looking at the operating system resource pages (free CPU, free memory) can show that there are available resources. However, these resources are being shared or constrained at a deeper level in the larger VM server setup. So, use VM tools as explained in the following technote.
- Initially I thought there is a WSDL caching mechanism involved when calling inbound WS within the BPM. So, I was really hoping that I could check numbers in instrumentation monitor page using extended page URL:
Using the above URL I have found the following section which was exactly what I was looking for but unfortunately it's NOT working in IBM BPM 8.5.x. E.g. no any numbers for either inbound or outbound WS calls are getting reflected/populated here, it just always stays at 0 ...
So, this is more for your awareness that this does not work unfortunately. And the reason I brought it up here anyway is because I started to investigate why it's not working and spent some time working through this with IBM L2/L3 Support and here is what I found out -
- 1) There is no WSDL cache for inbound web services in IBM BPM 85x. Period. As such, it does not make sense to expect the Instrumentation page to reflect anything for Inbound.
Instead, that's the process application version context cache. The PA version context cache is a small and unfortunately not adjustable cache. The WSDL is generated on every WSDL query call which is not cached.
- 2) For outbound web services, if you enable the "use-pre856-wsdl-cache" setting (in 100Custom.xml configuration), a non-persistent cache could be used, otherwise the WSDL would be loaded into the database ("local repository"). Even in this case, the Instrumentation is for persistent content cache used by web service. because WSDL cache is not a persistent content .. it's not mentioned. For outbound web services there was an engine switch in BPM 8.0 that completely redesigned the whole web service stack and since then the corresponding instrumentation data is not populated anymore. You can fall back to an old engine by turning the flag in 100Custom.xml to false but that is not recommended (<use-jaxws>true</use-jaxws>)
Improving the performance of web service connectors in IBM BPM
Luckily there is a number of ways how you can improve the performance of your web service connectors in IBM BPM.
- Simplify the WSDL and XSDs, i.e. merge multiple includes, flatten down the XSD (turn tree of XSD into single XSD).
- Cache the outputs of the SOAP calls, if the data is static enough
- Call the SOAP services via AJAX wrappers from Coach, instead of sequential invocation of the SOAP services before the coach, if requirements permit
- Make sure your WSDL cache configured and is sized accordingly. See the following doc.
- Nice overview of WSDL caching in IBM BPM 856+ in the following post.
- There is a section in IBM BPM Performance Redbook that is specific to web services tuning that you may want to pay attention to. It's on page 91, section 4.6.5. Copying that section here -
4.6.5 Web services tuning
If the target of the web services import binding is hosted locally in the same application server, you can further improve the performance by taking advantage of the optimized communication path that is provided by the web container.
Normally, requests from the web services clients are sent through the network connection between the client and the service provider. For local web services calls, however, WebSphere Application Server offers a direct communication channel, bypassing the network layer completely.
Complete the following steps to enable this optimization. Use the administrative console to change these values:
1. Click Application servers and the then the name of the server that you want.
Click Container Settings → Web Container Settings → Web container → Additional Properties → Custom Properties.
Set the web container custom property enableInProcessConnections to true.
Do not use wildcard characters (*) for the host name of the web container port.
Replace it with the host name or IP address.
The property can be accessed by clicking the following path: Application servers → messaging engine → Container Settings → Web Container Settings → Web container → Additional Properties → Web container transport chains → WCInboundDefault → TCP inbound channel (TCP_2) → Related Items → Ports → WC_defaulthost → Host.
2. Use localhost instead of host name in the web services client binding. Using the actual host name (even if it is aliased to localhost), disables this optimization.
To access the host name, use the following steps:
a. Click Enterprise Applications and select the application name.
b. Click Manage Modules and select the application EJB JAR file.
c. Click Web services client bindings → Preferred port mappings and select the binding name. Use the localhost (for example, localhost:9080) in the URL.
3. Make sure that there is not an entry for the server host name and IP address hosts file of your server for name resolution. An entry in the hosts file inhibits this optimization by adding name resolution processor usage.