OpenLM Performance Test

Scope

This document describes the performance tests done by OpenLM’s support team in order to help customers to choose the suitable configuration for their systems.

These tests evaluate the performance of the OpenLM system using 32 and 64 bit operating systems, different number of CPUs, different memory sizes and with external or internal databases.
The results of these tests promote a knowledge base that enables users to select the best database for their machine; A database that would maximize the performance of their OpenLM system.

Table of Contents:

OpenLM Performance Test

Scope:
Table of Contents:
Test environment
Test Data
The test stages:
Firebird DB (Embedded):
Operating System: Windows 7 32bit
Operating System: Windows 7 64bit
Microsoft SQL Server DB :
Operating System: Windows 7 32bit
Oracle:
Results:
Discussion and Recomendations

Test environment

The test was done using two computers with Intel I5 processor with 4 GB RAM. During the test the OpenLM Server was running on a virtual machine employing different amounts of memory and CPU.

When we tested the system using external databases, the external DB software (Oracle or MS SQL Server) ran on a different machine; not on the machine on which the OpenLM Server was run.

The queries were sent to the OpenLM Server virtual machine using a dedicated software developed for this purpose. This software was designed to send a set of queries to the OpenLM Server. It awaits acknowledgement, and calculates the elapsed time period.

The tests were conducted in the following environments:

1. Windows 7 32 bit operating system:
   1 CPU, 1 GB base memory
   1 CPU, 2 GB base memory
   2 CPUs, 1 GB base memory
   2 CPUs, 2 GB base memory
   4 CPUs, 1 GB base memory
   4 CPUs, 2 GB base memory

2. Windows 7 64 bit operating system:
   1 CPU, 1 GB base memory
   1 CPU, 2 GB base memory
   2 CPUs, 1 GB base memory
   2 CPUs, 2 GB base memory
   4 CPUs, 1 GB base memory
   4 CPUs, 2 GB base memory

Test Data

The test environment database is an actual customer’s database, that was selected as a test case. It contains information on more than 20 different license servers and vast number of stored license sessions. The magnitude of this database enables us to assess the performance of the OpenLM Server in big organizations.

The test benchmark data consists of 10 queries, submitted to the OpenLM Server. Each query fetches 100 records. The following set of queries was sent to the OpenLM server 3 times, sequentially:

1. Alerts query.

2. Denials query

3. Denials query with a specific feature.

4. License Activity report query

5. License Activity report query- fetching the information of a specific feature

6. License Activity report query- fetching the information of a specific server and vendor

7. License usage report query.

8. License usage report query – fetching the information of a specific vendor and feature.

9. Server statistics query.

10. Selected feature statistics query.

The test stages

The tests were conducted using various system configurations, evaluating the effect of different parameters on the system’s performance. The test variable parameters included:

• The Number of CPUs on the OpenLM server machine: 1, 2, and 4 CPUs.

• The Virtual Windows 7 operating system version: 32 bits Versus 64 bits.

• Different external and internal Databases were examined:

  • Internal database (OpenLM’s default DB): Firebird embedded DB.

  • External databases: MS SQL server, Oracle.

The test results

Firebird DB (Embedded):

Operating System: Windows 7 32bit

Operating System: Windows 7 64bit

Microsoft SQL Server DB :

Operating System: Windows 7 32bit

Operating System: Windows 7 64bit

Oracle DB:

Operating System: Windows 7 32bit

Operating System: Windows 7 64bit

The test limitations

The test is limited by a few factors, due to our ability to build a test environment:

1. Not a live system

We have used a big DB from a live system, but during the test OpenLM did not have to deal with routine tasks such as:

• • Receiving and processing broker information.

  • Query license server information and update DB tables.

  • Responding to Agent requests.

Please note that the OpenLM Server is a multi threaded system. Each of these tasks is processed in separate thread, so the effect of this factor is unknown.

2. Limited RAM configurations

We were limited to a single machine with a total of 4 GB of RAM. This limited our OpenLM Server machine memory allocation to 2 GB. We have experienced slight differences in performance between 1 and 2 GB RAM when using Oracle and MS-SQL, in favor of the 2GB configurations. Firebird DB demonstrated a big difference between 1GB and 2GB of RAM in favor of 2GB.

3. The limitation of the testing tool

The testing tool sequentially sends the same set of queries to the OpenLM Server. We have used 10 unique queries that were each sent 3 times to the server. This method may have created a diversion in favor of databases that have cache memories, such as Oracle and MS SQL Server.

Result Analysis

1. The results show significant differences in performance between the embedded Firebird database systems and external database systems such as MS-SQL and Oracle, in favour of the external databases. In some cases the external database performed in excess of 10 times faster than the embedded one.

2. The performance of MS-SQL was noticeably better than that of the other external database we have tested: the Oracle database.

3. We have witnessed a positive linkage between the Server’s performance and the allocated memory size: The performance of 2GB was generally better than that of 1GB. This is especially evident when referring to the embedded Firebird database.

4. On the embedded Firebird DB, we have seen improved performance when using the windows 64 bit OS, in comparison to 32Bit OS. This was not true in the case of external databases; Oracle and MS-SQL. The performance in the latter was better using a 32 bit OS.

5. The results were inconclusive in regards to the added value of additional CPUs on the test performance. There was an inclination towards a positive linkage between the number of CPUs and the performance in regards to embedded Firebird databases, when using a 64bit  OS.

Discussion and Recommendations

The tests we have performed are informative but limited due to the limitations listed in the limitation section. We would like to recommend the following guidelines to customers designing their system architecture:

The use of an external DB significantly contributes to the performance of the system. External database also provides multiple user access, scalability and other benefits. The downside of having an external DB is higher costs and need for a professional workforce that is required for maintaining such databases. When considering the MS-SQL database vs Oracle’s database, we have found the prior to perform better.

We recommend using extensive RAM; 2GB has performed better that 1GB.

Based on these tests, we are not able to recommend a 64 bit system over a 32 bit system. We think that the benefits of a 64 bit system may become significant when more than 4 GB or RAM is available. Because of that we prefer 64 bit operating system when available.