Skip to content

Introduction

OSCAR-logo

OSCAR is an open-source platform to support the event-driven serverless computing model for data-processing applications. It can be automatically deployed on multi-Clouds, and even on low-powered devices. It represents the porting to an on-premises scenario of the SCAR framework, which supports a High Throughput Computing Programming Model to create highly-parallel event-driven data-processing serverless applications that execute on customized runtime environments provided by Docker containers run on AWS Lambda.

Goal

Users upload files to a bucket and this automatically triggers the execution of parallel invocations to a function responsible for processing each file. Output files are delivered into an output bucket for the convenience of the user. Highly scalable HTTP-based endpoints can also be offered to expose a generic application. A user-provided shell script is executed inside the container run from the user-defined Docker image to achieve the right execution environment for the application.

Components

oscar arch

OSCAR runs on an elastic Kubernetes cluster that is deployed using:

  • IM, an open-source virtual infrastructure provisioning tool for multi-Clouds.

The following components are deployed inside the Kubernetes cluster in order to support the OSCAR platform:

  • CLUES, an elasticity manager that horizontally scales in and out the number of nodes of the Kubernetes cluster according to the workload.
  • MinIO, a high-performance distributed object storage server that provides an API compatible with S3.
  • Knative, a serverless framework to serve container-based applications for synchronous invocations (default Serverless Backend).
  • OSCAR Manager, the main API, responsible for the management of the services and the integration of the different components.
  • OSCAR UI, an easy-to-use web-based graphical user interface aimed at end users.

As external storage providers, the following services can be used:

  • External MinIO servers, which may be in clusters other than the platform.
  • Amazon S3, AWS's object storage service that offers industry-leading scalability, data availability, security, and performance in the public Cloud.
  • Onedata, the global data access solution for science used in the EGI Federated Cloud.
  • Any storage provider that can be accessible through WebDAV protocol. An example of a storage provider supporting this protocol is dCache, a storage middleware system capable of managing the storage and exchange of large data quantities.

Note: All of the mentioned storage providers can be used as output, but only MinIO can be used as input.

An OSCAR cluster can be easily deployed via the IM Dashboard on any major public and on-premises Cloud provider, including the EGI Federated Cloud.

An OSCAR cluster can be accessed via its REST API, the [web-based UI and the command-line interface provided by oscar-cli.