PET

PET

Performance Evaluation Tool

Performance Evaluation Tool (PET)


Overview

Performance measurements and simulations produce large amounts of data in a short period of time. Release cycles are getting shorter due to the DevOps movement and heavily rely on live data from production or test environments. In addition, performance simulations increasingly become accurate and close to exact predictions. Results from these simulations are reliable and can be compared with live data to detect deviations from expected behavior. PET is a tool that can process and analyze measurement as well as simulation data quickly utilizing big data technologies. Live measurement data and simulation results can be analyzed for detecting performance problems, deviations from expected behavior or to simply compare a performance model with real world applications.

PET Architecture

PET

Project Structure

The project consists of six plug-ins which are described in the following sections:

  1. org.fortiss.pmwt.pet.build
    This plug-in contains the parent POM file to build the entire project and all its artifacts.
  2. org.fortiss.pmwt.pet.csv.client
    This plug-in contains the CSV client to parse and store simulation results into the PET-Repository.
  3. org.fortiss.pmwt.pet.data
    This plug-in contains common utility functions and model files for all other plug-ins.
  4. org.fortiss.pmwt.pet.faban
    This plug-in contains the parser for the response times and run info from the Faban load test driver.
  5. org.fortiss.pmwt.pet.repository
    This plug-in contains the repository interface to store and load informations from and to the repository.
  6. org.fortiss.pmwt.pet.webApplication
    This plug-in contains the webservice and the REST-Interface to retrieve, visualize and analyze measurements and simulations stored in the repository.

Getting Started

  • Setup JDK (Version >= 1.7)
  • Setup Maven (Version >= 3.0)
  • Build PET with Maven. For example, run:
    cd org.fortiss.pmwt.pet.build/
    mvn clean install
    This command creates a new folder target in every plug-in which contains the artifacts.

Installation and Setup Repository Server

The PET-Repository server can be downloaded from the following link: PET-Repository
After unzipping the repository the repository can be started by calling ./<server_dir>/bin/cassandra
Out of the box the repository is configured to listen on the localhost address only. If you want to change this you have to perform the following changes in the configuration file. The configuration file can be found under /<server_dir>/config/cassandra.yaml:

  • Change listen_address from localhost to the address of your choice or use the listen_interface property to specify a interface of your choice. For example to listen on the interface eth0 use: listen_interface: eth0
  • Change start_rpc from false to true
  • Change rpc_address from localhost to to the address of your choice or use the rpc_interface property to specify a interface of your choice. For example to listen on the interface eth0 use: rpc_interface: eth0

For native clients the repository uses the port 9042 to change this adjust the property native_transport_port to the port of your choice. RPC is used for thrift clients and uses the default port 9160, to change this adjust the parameter rpc_port to the port of your choice.

Installation and Setup Web Application

Change to the directory org.fortiss.pmwt.pet.webApplication/target and deploy the PET.war file to a Wildfly server. After a successful deployment the webservice should be accessible under <wildfly_address>:<wildfly_port>/PET.

CSV Client

The CSV Client allows for parsing CSV simulation result files and storing them into the PET repository.

  1. Configurations
    CSV client can be configured using the command line interface.
    • Minimum configurations:
      -a,--repositoryAddress <REPOSITORY_ADDRESS> The address of the repository. -n,--repositoryName <REPOSITORY_NAME> The name of the repository. -c,--csvUri <CSV_URI> The URI to the directory with the CSV files from the simulation. -p,--pcmModelUri <PCM_MODEL_URI> The URI to the directory with the PCM model files. -sN,--simulationName <SIMULATION_NAME> The name of the simulation (Default: The name of the model directory).
    • Print all available configuration in your console:
      -h, --help Print this Message
  2. Run CSV Client
    Run from the target folder created by your maven build command
    • Run CSV Client and start parsing immediately: java -jar org.fortiss.pmwt.pet.csv.client-2.0.0.qualifier.jar -a 192.168.22.188 -n pmwt -c simulation_results -p pcm_models -sN sim_name
  3. Results
    CSV client will create (if not exists) a table called simulation, simulation_utilization, simulation_response_time and simulation_network_utilization to store simulation results and a table models to store the PCM models inside your repository.

Faban Client

The Faban Client allows for parsing CSV response time files and run info, generated from a Faban Load Test Run, and storing them into the PET repository.

  1. Configurations
    Faban client can be configured using the command line interface.
    • Minimum configurations:
      -a,--repositoryAddress <REPOSITORY_ADDRESS> The address of the repository. -n,--repositoryName <REPOSITORY_NAME> The name of the repository. -c,--csvUri <CSV_URI> The URI to the directory with the response time CSV files.
    • Optional configurations:
      -ws,--useWebService <TRUE/FALSE> Start Webservice? (default false) -wsPort,--webservicePort <PORT> Webservice Port (default 8080)
    • Print all available configuration in your console:
      -h, --help Print this Message
  2. Run Faban Client
    Run from the target folder created by your maven build command
    • Run Faban Client and start parsing immediately: java -jar org.fortiss.pmwt.pet.faban-2.0.0.qualifier.jar -a 192.168.22.188 -n pmwt -c faban_logs
    • Run Faban Client and start only webserver and webservice: java -jar org.fortiss.pmwt.pet.faban-2.0.0.qualifier.jar -a 192.168.22.188 -n pmwt -c faban_logs -ws true -wsPort 8080
      • Parse and persist response times to repository using configured CSV path: http:/localhost:8081/startPersistence
      • Parse and persist response times to repository using custom CSV path: http:/localhost:8081/startPersistence/{csvRootDir}
  3. Results
    Faban client will create (if not exists) a table called faban_response_time to store response times and a table faban_run_info to store run infos inside your repository.

SMT Client

The SMT Client allows for monitoring system information (e.g., CPU, Memory and HDD Metrics) in regular time intervals and storing them into a repository (e.g., Apache Cassandra).

  1. System setup
    • Setup JDK (Version >= 1.7)
    • Setup Maven (Version >= 3.0)
    • Build SMT with Maven. For example, run: cd smt/
      mvn clean install
  2. Configurations
    SMT can be configured using the command line interface.
    • Minimum configurations:
      -a, --address <REPOSITORY_ADDRESS> Repository Address -n, --name <REPOSITORY_NAME> Repository Name -e, --experimentName <EXPERIMENT_NAME> The experiment name of this measurements. -dRS,--diskReadSpeed <BYTES/S> The disk read speed in Bytes/s. -dWS,--diskWriteSpeed <BYTES/S> The disk write speed in Bytes/s.
    • Optional configurations:
      -i, --interval <INTERVAL> Measuring Interval in Seconds (default 10) -ws, --webservice <TRUE/FALSE> Start Webservice? (default false) -wsPort, --webservicePort <PORT> Webservice Port (default 9999) -d,--diskName <DISK_NAME> The disk name to monitor (default: /) -p,--processID <PROCESS_ID> The process ID to monitor (default: 0)
    • Print all available configuration in your console:
      -h, --help Print this Message
  3. Run SMT
    Run from the target folder created by your maven build command
    • Run SMT and start monitoring immediately: java -jar org.fortiss.pmwt.smt-1.0.jar -a 192.168.22.188 -n pmwt -e experimentName -ws false -wsPort 8081 -dRS 140000000 -dWS 120000000 -p 1000 -i 1
    • Run SMT and start only webserver and webservice: java -jar org.fortiss.pmwt.smt-1.0.jar -a 192.168.22.188 -n pmwt -e experimentName -ws true -wsPort 8081 -dRS 140000000 -dWS 120000000 -p 1000 -i 1
      • Start Monitoring by calling the webservice (e.g., http:/localhost:8081/startMonitoring)
      • Stop Monitoring by calling the webservice (e.g., http:/localhost:8081/stopMonitoring) (Note: The Webserivce keeps running and monitoring can be stopped and started again and again)
  4. Results
    SMT will create (if not exists) a table/keyspace called measurement inside your repository to store the measurements.