Release Summary 16.00.00

****Please reach out to Support before applying this update*****

v16.00.00 Major Update Prerequisites & Cautions

• Upgrade Notice: Please note that along with many new features and improvements, the major changes in v16 break some backwards compatibility! The following will be affected when the v16 update is applied:
• Services-related API endpoints; reference the breaking changes document for a complete list and new v2 API endpoints.
• Services-related DOQL views are affected; reference the breaking changes document for a list of affected views.
• Advanced Reporting: Any reports utilizing views listed in the breaking changes document will need to be updated.

The following tools and integrations MUST be updated along with the upgrade to v16:

• Netflow users must update their NetFlow collector to support collection of new statistics – see “Statistics section” of this document for details.
• The following integrations must also be upgraded to function with this release:
• Jira (self-hosted)
• ServiceNow – (available shortly; release still under review)
• Cherwell
• For full details around affected API endpoints and DOQL views, please refer to our breaking changes document. Expect a prolonged upgrade time for databases with large volumes of Service instances [see the end of this post for estimates].

What’s new in v16.00.00?

• Now that we’ve gotten the backwards compatibility announcement out of the way (you read the important info above, right!?), we’re very excited to announce the release of Device42 v16.00.00 for general availability — A milestone release chock-full of new features!
• v16 Affinity Groups with timelines
• To prepare for v16, Device42 developers gathered everything they’d learned over the years and popped the hood on the Services and Application Dependency Mapping (ADM) functionalities to determine how to best develop these features going forward.
• In doing so, it was clear that it was time to re-architect the modules – a major undertaking! Device42 v16 therefore marks an important milestone, introducing an all-new ADM and Services feature-set that will both better meet customers’ immediate needs and at the same time lays a better foundation to support future growth.
• Change highlights include:
• All-new, simplified approach to Services
• Affinity Groups Enhancements
• A variety of UX Improvements
• Packet Capture – A new discovery method!

Simplified Services

• Device42’s approach to Services was overhauled in v16, better aligning the module and its workflows to the way customers are using services in the real world. The net effects of these changes should equate to less duplicated data and better overall information. Working with Services should be easier, too.

Simplified Services Objects & Consolidated Service Instances

• Prior to v16, duplicate service instances were common due to variations in startup command line parameters (frequently seen with java-based applications). Device42 now applies new intelligence to the discovery of services, merging services that fall into this category together and storing any historical command line parameter variations within the details view.
• Going forward, there will be just one Service record for each service, representing all instances of a given service. Take Java as an example: prior to v16, you might see hundreds of ‘java’ services on each of your servers; going forward, you will have only one instance (per server).
• Additionally, the display name of each service is now customizable (e.g. you might want to rename several instances of ‘java’ to ‘MyCustomApplication’). Discovery will of course seamlessly match against your new name, prioritizing it for the import of the service instance’s command line arguments.

Introducing “Service Listening Ports” & “Service Communication” records

• Device42 v16 merges both Service Ports and Service Port IPs into a single object: Service Listening Ports. This combined object now holds only listener information and information about clients connected to each listener. Service objects are no longer created to represent applications and/or services doing client-only communication (e.g. internet explorer and similar).
• This information, however, is still captured and can still be seen in the newly introduced “Service Communication” records, which contain end-to-end information about client-to-service connections. Service Communication records have their own main menu item, which you can navigate to for details about both sides of the conversation — regardless of whether both sides have been discovered!

Undiscovered Listeners

• Starting with v16, Device42 identifies and surfaces undiscovered listeners that previously blended into service port data. Prior releases captured communications between servers known to D42 and unknown (undiscovered) receiving listeners, but those details were not displayed.
• Device42 now automatically creates an “Undiscovered Listener” to track these communications, and they are now also included in topology charts as a placeholder until subsequent discovery jobs discover them. After upgrading, you may notice that numerous Undiscovered Listeners were created from the historical data already in your system.

Scheduled tasks

• Device42 now separates scheduled tasks from services. Scheduled tasks are no longer directly related to services, and now have their own top-level menu item. Scheduled tasks also now have their own APIs and DOQL and are no longer tied to service instances.

Affinity Group Enhancements

• Affinity Groups received two important new UI features that should make them even more powerful than they already were: An all-new “Visual Timeline”, and “Affinity Group Calculation” views (more info on that below).
• Affinity Groups picked up a new “Configuration” button that makes it easy for users to customize Affinity Group processing, only generating Impact Groups for those users that don’t care to utilize dependency groups. Reference the screenshot to Preview the Affinity Groups Configuration!

Brand-new “Visual Timeline”

• While viewing Affinity Groups, Device42 now displays timestamps, clearly showing when each Affinity Group was changed (both at the device or the service level). This clearly displays when devices and/or services were added or removed from each Affinity Group.  The timeline makes it easy to see and quickly navigate to those specific dates when changes occurred, making it easy to see how an Affinity Group changed over time.
• This is one feature you’ll want to play around with!

Affinity Group Calculations

• Prior releases offered pre-packaged views into the calculation of affinity groups, namely, the ‘Affinity Group’ view. This release expands on available views, offering an exploratory view into the calculations. You can explore any device and can look in detail at its dependent and impacted servers (based on the Affinity Group Calculation).
• By default, Device42 will only show first-level connections, but you can easily expand each level to go deeper, inspecting 2nd, 3rd, and so-on levels deep. Note that the new “visual timeline” functionality is not available in exploratory mode; there are too many possibilities to calculate them all. Users can easily, however, utilize the ‘effective date’ to view a specific date’s historical data.

Multiple Application Components for Services

• Prior to v16 a given service instance could only be part of a single application component. This limitation often conflicted with real life use cases, where services often play a part across several different applications. This was commonly seen with load balancers, where a given backend server might be answering for multiple front-end application endpoints.
• This release alleviates these restrictions and includes updated visuals in support of these architectural changes. Application components will no longer be rendered as ‘container boxes’ but are now rather displayed as stand-alone nodes; dotted line(s) connect them to associated services and devices. These changes apply across the whole Device42 system.
• Note: Due to the required API and DOQL view modifications that support these changes, there will be some backwards-compatibility breakage as a result of these updates. See the breaking changes document for complete details.

Better Statistics

• Statistics were restructured to include sample counts as well as information about the number of times a connection was active for netstats. For Netflow, Device42 now includes details both about the time-span a given communication between two endpoints was active (and/or inactive). These statistics provide a clear picture of how consistently a given group of endpoints communicate.
• As a side effect of these drastic statistics changes (and to ensure displayed numbers accurately reflect counts going forward), as part of this update, running statistics must be reset; they are not, however, lost — The old numbers are preserved in what is now called the “classic values” view so that they can be referenced if needed. Additionally, we are now only keeping track of statistics from listener perspective, as those have proven to be the most useful (and most accurate) stats.
• Netflow collector users MUST update their NetFlow collector when they upgrade to v16 to support these restructured statistics!

Packet Capture – A new discovery method

• Device42 v16 debuts a brand-new way to gather connectivity data: Packet Capture. This new utility is currently compatible with any Windows or *nix-based machines, and can be customized to filter specific packets of interest.
• Device42 users can download the packet capture utility from device42.com and can run it on any supported device to capture detailed communication information flowing both in and out of the targeted network interface.
• Keep an eye out for a follow-up post discussing Packet Capture & Device42!

v16 Upgrade time estimates

• Expect a longer than usual upgrade time when deploying v16. Users with many service instances (reference chart below) might consider temporarily allocating extra RAM to their Device42 MA instance, which may help speed up the upgrade’s data migrations. PostgreSQL (Device42’s main data-store) loves RAM and will make use of all that is available within reason.
• Ensure you have reviewed the upgrade pre-requisites listed at the beginning of this post. V16 includes changes that break backwards compatibility!
• The following test data can serve as guidance when for upgrade planning; keep in mind, however, that your mileage may vary depending on the types of data stored in your specific instance & resources available to your MA — not all data is processed in the same way!
• Service Instances         Service Port Remote IPs                Tested Time
• 213,593                        1,282,673                                          0:10:47
• 336,610                        641,479                                             0:24:01
• 2,1898                          12,744,147                                        1:48:52
• Customers with large DBs (>5GB; check your DB size in Appliance Manager -> System -> Usage/Network Info) may want to contact support to schedule an assisted upgrade – email support@device42.com.

Link to our Blog - https://www.device42.com/blog/2019/08/better-services-management-packet-capture-and-more-in-milestone-release-v16-00-00/