apache/spark

Spark is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R (Deprecated), and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, pandas API on Spark for pandas workloads, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing. [](https://opensource.org/licenses/Apache-2.0) [](https://search.maven.org/search?q=g:org.apache.spark) [](https://adoptium.net/temurin/releases/?version=17) [](https://github.com/apache/spark/actions/workflows/buildmain.yml) [](https://codecov.io/gh/apache/spark) [](https://pypi.org/project/pyspark/) You can find the latest Spark documentation, including a programming guide, on the project web page. This README file only contains basic setup instructions. Spark is built using Apache Maven. To build Spark and its example programs, run: (You do not need to do this if you downloaded a pre-built package.) More detailed documentation is available from the project site, at "Building Spark". For general development tips, including info on developing Spark using an…