Skip to main content

jBEAM Cluster

Parallelize analyses with jBEAM Cluster
One Data Lake with multiple jBEAMs

jBEAM Cluster

Within one data lake several instances of jBEAM, which are located in a jBEAM cluster, analyze different measurement data simultaneously. The partial results will be aggregated to a final result.

jBEAM Cluster:

  • An analysis is created in one jBEAM and is processed in parallel in the jBEAM cluster with several jBEAMs.
  • The jBEAM Cluster consists of several PCs, on which one or more jBEAM instances (nodes) are run.
  • Each cluster Node processes one file, sends the result back to the aggregator and informs the cluster manager it is finished and ready for the next file.
  • This method corresponds to a MapReduce with files. Splitting the files is not necessary.
  • After all files are processed, the aggregator aggregates the single results and sends the end result to the user (client) via network.
  • Conventional file systems can be used (Windows, NAS, Linux, …) or the Hadoop Distributed File System (HDFS).

Transferring vast amount of data will be unnecessary

No raw data is transferred to the user PC. Only the end result is send back to the user.

Calculations are brought to the data

Calculation will be run where the data is actually located.

Integrable into the measurement data management system MaDaM

For desktop applications as well as for global MDM systems.

Components for Cluster Operation:

  1. File Importer
    A file importer defines, how to import the files (jBEAM supports more than 100 data formats)
  2. Data Reduction Calculations
    Statistical (Min, Max, …), Event-Detection, Histogram, Rainflow, …
  3. Aggregator
    Component, which defines how the results should be aggregated (sum, append, …)
  4. Multi-File-Analysis-Controller (MFAC)
    jBEAM side controller to create cluster jobs
  5. Cluster Service
    There are different types of cluster services

    • One node
      Sequential operation
    • Local Cluster
      jBEAM instances in own Java VM
    • External Cluster
      Built with several PCs, each with one or N nodes (jBEAMs)
    • External High Performance Cluster
      Plenty of high performance Linux PCs

More Information about jBEAM Cluster