Evaluation Criteria

Performance and support evaluation of hosting service providers is based on a set of tests and criteria to avoid inaccuracy in marks. Below you can find out what was considered while evaluation. The recalculations of the results are performed once in a quarter.


Support

Support evaluation is performed by Jelastic team and end-customers. Initially, we perform Jelastic evaluation based on the following criteria:

  • reaction on monitoring system (resolution of disasters and high-priority situations that can influence end users): 3 points
  • complaints of end customers about the quality of hoster’s support directly to Jelastic team: 3 points
  • 24/7 support and hotline number for urgent cases: 4 points
  • 1st, 2nd, 3rd line of support: 1, 2, 3 points for each level accordingly
  • easy entry point to hosters support portal through dashboard (working links, SSO, intuitive UI of support portal): 2 points
  • SLA: 2 points
  • support of the latest Jelastic versions: 2 points
  • availability of support for trial users: 1 point

General mark for the support services is a sum of the points for every above stated item:

  • 18 – 20 points
  • 15 – 17 points
  • 10 – 14 points
  • 5 – 9 points
  • 0 – 4 points

The newly launched hosting partners are marked with icon and evaluated with marks during the first half of the year after commercial release.


Performance Benchmark Procedure

The benchmark procedure is conducted by Jelastic team in order to collect and analyze data on all partners Jelastic installations’ efficiency to define and assign an appropriate performance rate to each Platform. Such comparative analysis is performed by means of Phoronix Test Suite – a comprehensive testing and benchmarking platform with a dedicated framework, which could be easily extended with your own custom tests. This software is designed to effectively carry out both qualitative and quantitative benchmarks in a reproducible and easy-to-use fashion.

You can run Phoronix Test Suite by your own whenever you want to compare your Platform score with the published rate limits for every level. For that, perform the steps below.

Tip for Partners: If you have noticed that your cluster performance rate differs from the published one, contact your Technical Account Manager to recheck and update your Jelastic installation’s rate with proper values.
  1. Log in to your Dashboard as a billing user using your email and password.
    Dashboard
  2. To initiate the tests, import the manifest using the link: https://github.com/jelastic-jps/phoronix/blob/master/manifest.jps
    import
  3. If required, change the Phoronix software version, environment name, and preferable region.
    change the Phoronix software version
  4. Right after the environment is created the benchmark software runs:
    • 3 test flows for CPU
    • 6 tests for Disk I/O
    • 10 tests for RAM
  5. When the tests are completed, average values for the mentioned parameters will be provided in the success popup:
    • CPU <Time> value indicates how much time was spent to perform the test task
    • RAM <Speed> value represents memory throughput as a number of Megabytes per second
    • HDD <Speed> value reflects disk subsystem speed known as IOPS (Input/Output Operations per Second)

    the success popup

    Note: The result will be duplicated in the appropriate email and in the file /var/www/html/result.html that can be opened with configuration file manager at the Elastic VPS node.
  6. To calculate the overall Platform performance RATE (number of “stars”), use the following formula:RATE = Rate(CPU)*0.7+Rate(RAM)*0.15+Rate(HDD)*0.15Where the Rate(CPU), Rate(RAM) and Rate(HDD) values can be determined using the table below. The first row is the Rate() that should be used in the formula and the first column lists the parameters.
    2 3 4 5
    CPU, sec >154 122-155 93-123 <92
    RAM, MB/sec <10084 10085-14051 14052-18017 >18018
    HDD, MB/sec <63 64-123 124-183 >184

    Based on our sample test results (CPU 92 = 5, RAM 14052 = 4, HDD 64 = 3), the final formula looks as follows:

    RATE = Rate(5)*0.7+Rate(4)*0.15+Rate(3)*0.15 = 4.55

    Number of Stars for performance can be checked for each Hosting Provider at Jelastic Cloud Union catalog.
    Jelastic Cloud Union catalog