NUVOLOS
Sign In
  • Getting Started
    • Introduction to Nuvolos
    • Documentation structure
    • Nuvolos basic concepts
      • Organisational hierarchy
      • Applications
      • Distribution
      • Data integration
      • Snapshots
      • Background tasks
    • Navigate in Nuvolos
    • Quickstart tutorials
      • Research
      • Education (instructor)
      • Education (student)
  • Features
    • Applications
      • Application resources
      • Sessions
        • Session Logs
      • Install a software package
      • Create a persistent .bashrc
      • Automatic code execution
      • Long-running applications
      • Troubleshooting applications
      • New applications or licenses
      • Configuring applications
      • Exporting applications
      • Add-ons
        • MariaDB add-on
        • PostgreSQL add-on
        • OpenSearch add-on
        • MongoDB add-on
        • Redis add-on
        • PostGIS add-on
        • Rclone mount add-on
        • Neo4j add-on
    • File system and storage
      • File navigator
      • Large File Storage
      • Preview files
      • Mount Dropbox
      • Access S3 buckets with RClone
      • Access remote files with SSHFS
      • Access files on SharePoint Online
    • Object distribution
      • Distribution strategies
      • The distributed instance
    • Snapshots
      • Create a snapshot
      • Restore a snapshot
      • Delete a snapshot
    • Database integration
      • Create datasets
      • View tables
      • Build queries
      • Upload data
      • Access data from applications
        • Set up ODBC drivers
        • Obtain tokens for data access
        • Find database and schema path
      • DBeaver integration
    • Environment variables and secrets
    • Searching
      • Page
      • Find an application
      • Find an organisation
      • Find a space
      • Find an instance
      • Find a state
    • Video library
    • Nuvolos CLI and Python API
      • Installing the CLI
      • Using the CLI
  • User Guides
    • Research guides
      • Inviting a reviewer
      • GPU computation
    • Education guides
      • Setting assignments
        • Programmatical assignment handling
      • Documenting your course
      • Setting up group projects
        • Collaborative application editing
      • Configuring student applications
      • Archiving your course
      • Student guides
        • Joining a course
        • Working on assignments
        • Leaving a course
    • Application-specific guides
      • JupyterLab
      • RStudio
      • VSCode
      • Stata
      • MATLAB
      • Terminal
      • Terminal [tmux]
      • Apache Airflow
      • Apache Superset
      • D-Wave Inspector
      • MLFlow
      • Databricks Connect
      • Dynare.jl
      • CloudBeaver
      • InveLab
      • Overleaf
      • Metabase
      • DNDCv.CAN
      • OpenMetaData
      • Uploading data to the Large File Storage
    • Data guides
      • Setting up a dataset on Nuvolos
      • Importing data on Nuvolos
      • A complete database research workflow (Matlab & RStudio)
      • Accessing data as data.frames in R
      • Working with CRSP and Compustat
      • Working with the S&P 500®
  • Pricing and Billing
    • Pricing structure
    • Resource pools and budgets
    • Nuvolos Compute Units (NCUs)
  • Administration
    • Roles
      • Requesting roles
    • Organisation management
    • Space management
      • Invite to a space
      • Revoke a space user
      • HPC spaces
      • Resting spaces
    • Instance management
      • Invite to an instance
    • Enabling extra services
    • Monitoring resource usage
  • Reference
    • Application reference
      • InveLab
        • Dataset selection
        • Modules
          • Time-series visualisation
          • Moment estimation
          • Mean-variance frontiers
          • Frontiers
          • Dynamic strategy
          • Portfolio analysis
          • Performance analysis
          • Benchmarking
          • Carry trade strategies
          • Risk measures
          • Conditional volatility
          • Replication
          • Factor factory
          • Factor tilting
          • Valuation
    • Glossary
  • FAQs
    • FAQs
    • Troubleshooting
      • Login troubleshooting
        • I forgot my email address
        • I forgot my identity provider
        • I can't log in to Nuvolos
        • I forgot my password
        • I haven't received the password reset email
        • I haven't received the invitation email
      • Application troubleshooting
        • I can't see an application
        • I can't start an application
        • I can't create an application
        • I can't delete an application
        • I can't stop a running application
        • JupyterLab 3 troubleshooting
        • Spyder 3.7 troubleshooting
      • Administration troubleshooting
        • I can't see a space
        • I can't create a space
        • I can't delete a space
        • I can't invite admins to my space
        • I can't see an instance
        • I can't create an instance
        • I can't delete an instance
        • I can't invite users to an instance
        • I can't see distributed content in my instance
        • I can't see a snapshot
        • I can't create a snapshot
        • I can't delete a snapshot
        • I can't revoke a user role
        • I can't upload a file
        • I can't delete a file
        • I can't invite students to my course
      • Content troubleshooting
        • I can't find my files in my Linux home
        • I can't find my files among the Workspace files
        • I restored a snapshot by mistake
Powered by GitBook
On this page
  • Uploading data programmatically
  • Python
  • R
  • Matlab
  • Using Excel
  • Support request

Was this helpful?

  1. Features
  2. Database integration

Upload data

Nuvolos users can import and upload data into their tables. To do so, the user has a number of options which are illustrated below.

Before proceeding, please consider if your use-case is well suited for loading table data into Nuvolos. Table data has a number of advantages, especially in the following use-cases:

  • Combining with other datasets: if you are combining a dataset with other datasets on Nuvolos that come in a table format, it is often easiest to do the joins directly via SQL statements.

  • Cross-application compatibility: tables provide a cross-application typed data structure that can make data easier to analyse with multiple applications.

  • Large dataset support: tables support data volumes that are sometimes difficult to efficiently store in regular files.

If none of the above benefits are essential, then storing data in files is often the simpler solution.

Uploading data programmatically

The examples below assume that the user is uploading tables to Nuvolos using Nuvolos Applications. Make sure that the table file has been uploaded to Nuvolos. Check the documentation here for information on file upload.

Python

Instead of Pandas's df.to_sql, use nuvolos.to_sql , which has a similar signature but uses highly efficient bulk data loading. df.to_sql is not supported in Python applications running on Nuvolos.

We strongly recommend using lowercase Pandas DataFrame column and index names and a lowercase table name when uploading tables to Nuvolos in order for the resulting database table name and column name to be case-insensitive.

Using uppercase or mixed-case DataFrame column or index names can lead to inconsistent DataFrame column names when reading data with pandas.read_sql, depending on whether the code is run on- or off-Nuvolos.

from nuvolos import get_connection, to_sql
import pandas as pd

df = pd.read_csv(...) #read your data
con = get_connection()
to_sql(df=df, name="table_name", con=con, if_exists='replace', index=False)
from nuvolos import get_connection, to_sql
import pandas as pd

df = pd.read_csv(...) #read your data
con = get_connection(dbname = "dbname", schemaname="schemaname")
to_sql(df=df, name="table_name", con=con, if_exists='replace', index=False)

The above example assumes the use of the latest (>= 0.4.1) version of thenuvolos-odbc (for use in Nuvolos applications) or nuvolospackage (for off-Nuvolos use on your local PC) from PyPI.

To update to the latest nuvolos-odbc (for use in Nuvolos applications) or nuvolos package (for off-Nuvolos use), please execute the following commands in your Python environment:

In existing, older Nuvolos Python applications:

pip uninstall nuvolos
pip install --upgrade nuvolos-odbc

Off-Nuvolos, on your local PC:

pip install --upgrade nuvolos

R

We provide two solutions to load data into Nuvolos. The standard solution relies on the DBI::dbWriteTable call, which is sufficient for smaller tables. For larger tables, we suggest using an alternative method, as it provides significantly better performance and other convenience features.

Loading smaller tables:

df <- read.csv('path_to_data') #read your data
con <- nuvolos::get_connection()
DBI::dbWriteTable(con, name="table_name", value=df)

Loading large tables:

# Update the Nuvolos connector to the latest version
options(repos = "https://cran.rstudio.com")
install.packages("remotes")
remotes::install_github("nuvolos-cloud/r-connector")
use_condaenv(condaenv = "base", conda = "/usr/local/lib/conda/bin/conda")

nuvolos::to_sql(df=df, name="table_name", if_exists='replace', index=FALSE)
df <- read.csv('path_to_data') #read your data

# Update the Nuvolos connector to the latest version
options(repos = "https://cran.rstudio.com")
install.packages("remotes")
remotes::install_github("nuvolos-cloud/r-connector")
reticulate::install_miniconda()

# You need to restart your R session if miniconda was installed on line 7 and not before

nuvolos::to_sql(df=df, name="table_name", dbname="dbname", schemaname="schemaname", if_exists='replace', index=FALSE)

Matlab

df = readtable('path_to_data')
con = get_connection()
sqlwrite(con,'TABLE_NAME',df)

Using Excel

Support request

If you have a specific data onboarding task that you want to discuss with us, you can open a support request by sending an email to support@nuvolos.cloud.

PreviousBuild queriesNextAccess data from applications

Last updated 4 months ago

Was this helpful?

If you wish to upload data to Nuvolos from your own PC, please follow the and the database name and schema name to use:

If you wish to upload data to Nuvolos from your own PC, please follow the and the database name and schema name to use:

To upload data to Nuvolos using Excel, please refer to our instructions on

instructions on how to obtain connection tokens
instructions on how to obtain connection tokens
accessing and uploading data with Excel.