Upload table data to Nuvolos
Nuvolos users can import and upload data into their tables. To do so, the user has a number of options which are illustrated below.
Before proceeding, please consider if your use-case is well suited for loading table data into Nuvolos. Table data has a number of advantages, especially in the following use-cases:
- Combining with other datasets: if you are combining a dataset with other datasets on Nuvolos that come in a table format, it is often easiest to do the joins directly via SQL statements.
- Cross-application compatibility: tables provide a cross-application typed data structure that can make data easier to analyze with multiple applications.
- Large dataset support: tables support data volumes that are sometimes difficult to efficiently store in regular files.
If none of the above benefits are essential, then storing data in files is often the simpler solution.
​
1. Uploading data programmatically
The examples below assume that the user is uploading tables to Nuvolos using Nuvolos Applications. Make sure that the table file has been uploaded to Nuvolos. Check the documentation here for information on file upload.
1. Python
Instead of Pandas's
df.to_sql, use nuvolos.to_sql , which has a similar signature but uses highly efficient bulk data loading. df.to_sql is not supported in Python applications running on Nuvolos. We strongly recommend using lowercase Pandas DataFrame column and index names and a lowercase table name when uploading tables to Nuvolos in order for the resulting database table name and column name to be case-insensitive.
Using uppercase or mixed-case DataFrame column or index names can lead to inconsistent DataFrame column names when reading data with
pandas.read_sql, depending on whether the code is run on- or off-Nuvolos. ​
1
from nuvolos import get_connection, to_sql
2
import pandas as pd
3
​
4
df = pd.read_csv(...) #read your data
5
con = get_connection()
6
to_sql(df=df, name="table_name", con=con, if_exists='replace', index=False)
Copied!
If you wish to upload data to Nuvolos from your own PC, please follow the instructions on how to obtain connection tokens and the database name and schema name to use:
1
from nuvolos import get_connection, to_sql
2
import pandas as pd
3
​
4
df = pd.read_csv(...) #read your data
5
con = get_connection(dbname = "dbname", schemaname="schemaname")
6
to_sql(df=df, name="table_name", con=con, if_exists='replace', index=False)
Copied!
The above example assumes the use of the latest (>= 0.4.1) version of the
nuvolos-odbc (for use in Nuvolos applications) or nuvolospackage (for off-Nuvolos use on your local PC) from PyPI.To update to the latest
nuvolos-odbc (for use in Nuvolos applications) or nuvolos package (for off-Nuvolos use), please execute the following commands in your Python environment:In existing, older Nuvolos Python applications:
1
pip uninstall nuvolos
2
pip install --upgrade nuvolos-odbc
Copied!
Off-Nuvolos, on your local PC:
1
pip install --upgrade nuvolos
Copied!
2. R
We provide two solutions to load data into Nuvolos. The standard solution relies on the
DBI::dbWriteTable call, which is sufficient for smaller tables. For larger tables, we suggest using an alternative method, as it provides significantly better performance and other convenience features.Loading smaller tables:
1
df <- read.csv('path_to_data') #read your data
2
con <- nuvolos::get_connection()
3
DBI::dbWriteTable(con, name="table_name", value=df)
Copied!
Loading large tables:
1
# Update the Nuvolos connector to the latest version
2
options(repos = "https://cran.rstudio.com")
3
install.packages("remotes")
4
remotes::install_github("nuvolos-cloud/r-connector")
5
reticulate::install_miniconda()
6
​
7
# You need to restart your R session if miniconda was installed on line 5 and not before
8
​
9
nuvolos::to_sql(df=df, name="table_name", if_exists='replace', index=FALSE)
Copied!
If you wish to upload data to Nuvolos from your own PC, please follow the instructions on how to obtain connection tokens and the database name and schema name to use:
1
df <- read.csv('path_to_data') #read your data
2
​
3
# Update the Nuvolos connector to the latest version
4
options(repos = "https://cran.rstudio.com")
5
install.packages("remotes")
6
remotes::install_github("nuvolos-cloud/r-connector")
7
reticulate::install_miniconda()
8
​
9
# You need to restart your R session if miniconda was installed on line 7 and not before
10
​
11
nuvolos::to_sql(df=df, name="table_name", dbname="dbname", schemaname="schemaname", if_exists='replace', index=FALSE)
Copied!
3. Matlab
1
df = readtable('path_to_data')
2
con = get_connection()
3
sqlwrite(con,'TABLE_NAME',df)
Copied!
2. Using Excel
To upload data to Nuvolos using Excel, please refer to our instructions on accessing and uploading data with Excel.​
3. Support request
If you have a specific data onboarding task that you want to discuss with us, you can open a support request by sending an email to [email protected]
Last modified 9mo ago