In this article, I'd like to focus on those Camunda clients who are running on Camunda 8.x self-managed and considering an update (from 8.1 to 8.2 for instance). Information in this article is based on BP3 Production Operations team's experience with our clients whom we helped with such updates.
Without further ado let's get into it.
What are the important bits I need to take into consideration when updating Camunda 8.x?
One may tell that reviewing the official Camunda documentation (for example how to update from 8.1 to 8.2) might be sufficient but from our experience, there is more to it especially if you're already running in Production on Camunda 8. So, the assumption here going forward is that you reviewed the official documentation and understand what the upgrade steps may look like.
And instead, we will focus on those important considerations that are not necessarily directly Camunda-related but rather all the involved bits and pieces related (like Elasticsearch etc) making sure that the update process goes smoothly from an end-user/business perspective.
With that said, here's a high-level list of important considerations that you may want to take into account when planning such updates:
- Document high-level upgrade steps (action plan)
- Define and document prerequisites, these may include:
- ElasticSearch (what are the indices that will be reindexed, amount of data stored in the indices, reindex time consideration, cleanup, adjusting indices configuration to speed up the upgrade performance, analyze resources)
- Backup management
- Operate backup
- Zeebe backup
- Zeebe gateway management during the update process
- Offline vs online backup and so on
- Review and adjust configurations for optimal upgrade experience - Zeebe gateway, Zeebe, Operate
- Perform dry run upgrade, e.g. run a production-scale upgrade in a Sandbox type of environment using a replica of data from Production (this way you can get an idea on the down times required as well as verification of an overall update process)
- Test/verify the restore procedure (it's important to test if the defined backup procedure actually works)
- Plan and consider downtime. There are various factors that come into play here not just the downtime itself but the sequence of a shutdown, and the involved components. For example, everything can fall apart if one of the brokers will be updated and write an event with a new version in metadata and another broker will do with an old version.
Self-managed Camunda 8 environments might have different deployments, configurations, and overall infrastructure. As such, in this article, I wanted to focus on what you need to consider when performing such updates. Behind each of the items listed above, there could be a number of steps/procedures on how to actually implement it.
Feel free to contact BP3 Production Operations team at email@example.com for further assistance.