XONAI accelerates Spark jobs in your existing cloud data platform or on-premises environment. Activate today without code changes or migrations.
Performance benchmark
With XONAI
Up to 3X faster
XONAI delivers market-leading TPC-DS and TPC-H price-performance in any existing managed data platform for the cloud and without breaking runtime compatibility.
Price reduction
With XONAI
XONAI delivers consistent data processing acceleration in any hardware of choice and without any drawbacks.
Speedup of existing cloud big data platforms when XONAI is activated.
3TB TPC-DS and TPC-H benchmark summary on 5 workers on the respective hardware and instance. Source
XONAI transparently optimizes the Spark cache mechanism and delivers up to 6x cache performance with zero code changes.
Cache acceleration
Spark
Benchmarked with 300GB TPC-DS and 50GB TPC-H data sets (higher is better)
Our optimized columnar data processing format allows up to 3x better cache compression while delivering significant speedups in any compression scheme.
Cache storage size
Spark
Uncompressed offers top performance at the expense of more storage
Benchmarked with 300GB TPC-DS and 50GB TPC-H data sets (lower is better)
No code changes
XONAI activates at the layer closest to the hardware and does not compete with existing runtimes but augments them instead with hardware acceleration.
Our engine routes physical query plans to our novel MLIR-based compiler and execution engine purpose-built for accelerating data processing.
No migrations
XONAI is 100% API-compatible with Spark runtimes in all popular big data platforms for the cloud.
Our solution delivers consistent acceleration of Spark jobs in any platform and hardware. Source
The leading engine
Coming soon!
A plug-and-play Grafana-based UI that connects with your cloud and enables visibility over cloud spending for all your Spark jobs, identifies inefficient spending and assesses cost and performance optimization opportunities with XONAI even before activating it. Our engine provides detailed job execution metrics not available on any other Spark runtime.
Gain detailed visibility over cost and performance metrics of EMR clusters.
Understand how XONAI reduces Spark job costs and improves resource utilization.
See detailed execution and performance metrics unlocked by our engine for Spark applications.
Gain detailed visibility over cost and performance metrics of EMR clusters.
Understand how XONAI reduces Spark job costs and improves resource utilization.
See detailed execution and performance metrics unlocked by our engine for Spark applications.
XONAI for Apache Spark
Our solution integrates with the open-source Apache Spark 3 distribution and the following data platforms:
- Amazon EMR starting from 6.3.0
- Databricks up to 10.4 (preview)
- Dataproc 2.0.X and 2.1.X release line of versions (preview)
The solution is activated by a Spark 3 plugin which runs physical plans equivalent to the ones selected by Spark runtimes. In practice, the spark-submit command will point to a JAR provided by us via spark.plugins property.
Additionally, our engine requires moving a fraction of the spark.executor.memory to the spark.executor.memoryOverhead setting. This change is needed because our engine allocates off-heap memory to process data rather than JVM memory.
Existing solutions tackle cloud spending reduction by improving resource provisioning and/or tuning application parameters, and may have only a one-time benefit only for workloads not being optimally deployed.
Our solution accelerates Spark data processing speed far beyond the default Spark engine (Catalyst), and delivers hardware acceleration and reduced resource utilization regardless of how optimally deployed Spark workloads already are.
No. We intentionally designed our engine to be API-compatible with existing runtimes for Spark, including proprietary ones that may modify query plans to improve performance, such as the EMR runtime.
The more time queries spend on doing physical computations, the more benefit they are expected to get. These are typically queries with heavy aggregations, joins and sorting stages.
A drop-in solution that can be activated in your cloud environment with no code changes to reduce cloud costs and accelerate insight delivery.
Building a universal compute fabric to power data infrastructure worldwide.
