Built for solving big challenges with big data

Upsolver has what’s required to implement a continuous, at-scale, production pipeline. You can monitor pipelines, manage pipeline changes methodically, deploy in your VPC or our managed environment based on your requirements.

Monitoring

Upsolver lets you monitor your pipelines in our Visual SQL IDE or through integration with an external monitoring system.

Metrics that are monitored include data source volume, latency and errors; output volume, output volume and latency, compute cluster resource utilization and Upsolver Lookup Table volume, latency and query hits/misses.

These metrics can be viewed in the Upsolver app or sent to a 3rd-party monitoring system, such as DataDog, InfluxDB, Elasticsearch, Amazon CloudWatch or SignalFX. Upsolver will add integrations to other systems upon request.
Learn more about monitoring systems.

CI/CD

Upsolver provides complete separation between your development and production environments, which can be applied to all the entities configured in your Upsolver account.
When you are done developing and testing your pipelines in your development environment, deploying it to production takes just a few clicks. You use the identical pipeline you already tested and developed in your dev environment and run it on top of your production data streams.
Backing up your pipeline code is done using Upsolver’s Git integration feature. With one click, integrate Upsolver with your Git repository and have all Upsolver configurations, objects, and metadata sync to your repository in real-time. Using this functionality allows you to use all the familiar Git capabilities such as source code version management, collaboration, and code ownership.
Also, Upsolver provides a REST API that enables you to manage all Upsolver’s infrastructure from your code. This allows you to perform any operation you can perform in Upsolver’s IDE using our API.

Flexible cloud deployment options

Upsolver provides two deployment options: Private VPC and Fully Managed. In both Upsolver deployment options, the data is retained in the customer’s storage, and Upsolver hosts the control plane. The difference is in where the processing occurs, either in Upsolver’s account (Fully Managed deployment) or the customer’s VPC (Private VPC Deployment).

Private VPC Deployment

Private VPC mode gives you full security control over the data plane and ensures data never leaves your AWS account. Use your own VPC in AWS, or Upsolver’s fully-managed VPC.
Upsolver most commonly is deployed in Private VPC mode. Data processing occurs in the customer’s compute cluster and the data remains within the customer’s control. The control plane that ensures proper operation of the data plane runs in Upsolver’s account.
We recommend the customer deploy Upsovler to a new VPC in which case AWS CloudFormation will execute the deployment. An existing VPC can also be deployed but requires additional security configurations.

The benefits of the Private VPC approach are

• Data never egresses from customer account, avoiding compliance or policy violations.
• The customer benefits from any pricing discounts they have negotiated with their cloud vendor.
• The data plane is still managed by Upsolver, so that the customer doesn’t need to monitor Upsolver availability or reliability directly.

Fully Managed Deployment

You can utilize Upsolver as a fully managed service where Upsolver accesses data stored in a customer’s S3 buckets but runs processing on the Upsolver account. The data and data access is granted from outside of the user’s account. Upsolver processes the data and writes it back to the user’s S3 bucket.

One advantage of this deployment model is that it allows you to get up and running quickly.

Security

Regardless of whether you choose to deploy Upsolver on your virtual private cloud or ours, Upsolver clusters read data from your S3 bucket, process it in-memory and write back to the S3 bucket. We never store any of your data, but you can choose where our processing, serving and streaming clusters will reside.

When deploying Upsolver in Private VPC, it is deployed entirely in your AWS account. In this option, Upsolver is unable to access any of your data, and is restricted to managing compute clusters on your AWS account.

The Upsolver web interface (accessible via app.upsolver.com) is hosted on a public Amazon S3 bucket. It connects to the public Upsolver API, which redirects it to the private API that is hosted on your AWS account. The public API is unable to access any data in your account.

Upsolver creates two separate roles in your account:

Management role – can manage servers but cannot access data. This role can be assumed by Upsolver’s AWS account for troubleshooting or configuration purposes.

Data role – has access to data in S3 and Kinesis. This role is only assumable within your AWS account and cannot be assumed by Upsolver.

You control who can access your account by defining allowed and disallowed IP addresses. You can choose to permit Upsolver’s support team to access your web UI instance for troubleshooting and configuration purposes.

What internet traffic does the API use?

In private VPC, Upsolver does not require any inbound internet access. The private API does require outbound internet access, used for the following:

• Reading system configuration details
• Sending system metrics to Upsolver to ensure uptime and availability
• Sending logs (optional)
• AWS SDK calls (Athena, S3, Glue)

How does Upsolver handle troubleshooting and support?
Troubleshooting is done based on system logs and metrics. Upsolver support might ask for additional information or access, which the customer can choose to share but is not obligated to do so. Access can be granted and removed instantly by modifying the security group IPs.