| Title: | Download or Upload 'Socrata' Data Sets |
|---|---|
| Description: | Provides easier interaction with 'Socrata' open data portals <https://dev.socrata.com>. Users can provide a 'Socrata' data set resource URL, or a 'Socrata' Open Data API (SODA) web query, or a 'Socrata' "human-friendly" URL, returns an R data frame. Converts dates to 'POSIX' format and manages throttling by 'Socrata'. Users can upload data to 'Socrata' portals directly from R. |
| Authors: | Hugh Devlin, Ph. D., Tom Schenk, Jr., Gene Leynes, Nick Lucius, John Malc, Mark Silverberg, and Peter Schmeideskamp |
| Maintainer: | "Gene Leynes" <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.7.15-1 |
| Built: | 2025-03-03 04:48:39 UTC |
| Source: | https://github.com/Chicago/RSocrata |
Convert Socrata human-readable column name, as it might appear in the first row of data, to field name as it might appear in the HTTP header; that is, lower case, periods replaced with underscores#'
fieldName(humanName)fieldName(humanName)
humanName |
- a Socrata human-readable column name |
Socrata field name
Hugh J. Devlin, Ph. D. [email protected]
fieldName("Number.of.Stations") # number_of_stationsfieldName("Number.of.Stations") # number_of_stations
Will check the validity of a potential dataset unique identifier supported by Socrata. It will provide an exception if the syntax does not align to Socrata unique identifiers. It only checks for the validity of the syntax, but does not check if it actually exists.
isFourByFour(fourByFour)isFourByFour(fourByFour)
fourByFour |
- a string; character vector of length one |
TRUE if is valid Socrata unique identifier, FALSE otherwise
Tom Schenk Jr [email protected]
List datasets available from a Socrata domain
ls.socrata(url)ls.socrata(url)
url |
- A Socrata URL. This simply points to the site root. |
an R data frame containing a listing of datasets along with various metadata.
Peter Schmiedeskamp [email protected]
## Not run: # Download list of data sets df <- ls.socrata("https://soda.demo.socrata.com") # Check schema definition for metadata attributes(df) ## End(Not run)## Not run: # Download list of data sets df <- ls.socrata("https://soda.demo.socrata.com") # Check schema definition for metadata attributes(df) ## End(Not run)
Convert Socrata calendar_date string to POSIX
posixify(x)posixify(x)
x |
- character vector in one of two Socrata calendar_date formats |
a POSIX date
Hugh J. Devlin, Ph. D. [email protected]
Manages throttling and POSIX date-time conversions
read.socrata( url, app_token = NULL, email = NULL, password = NULL, stringsAsFactors = FALSE )read.socrata( url, app_token = NULL, email = NULL, password = NULL, stringsAsFactors = FALSE )
url |
- A Socrata resource URL, or a Socrata "human-friendly" URL, or Socrata Open Data Application Program Interface (SODA) query requesting a comma-separated download format (.csv suffix), May include SoQL parameters, but is assumed to not include a SODA offset parameter |
app_token |
- a string; SODA API token used to query the data portal https://dev.socrata.com/consumers/getting-started.html |
email |
- Optional. The email to the Socrata account with read access to the dataset |
password |
- Optional. The password associated with the email to the Socrata account |
stringsAsFactors |
- Optional. Should character columns be converted to factor (TRUE or FALSE)? |
an R data frame with POSIX dates
Hugh J. Devlin, Ph. D. [email protected]
## Not run: # Human-readable URL: url <- "https://soda.demo.socrata.com/dataset/USGS-Earthquakes-for-2012-11-01-API/4334-bgaj" df <- read.socrata(url) # SoDA URL: df <- read.socrata("https://soda.demo.socrata.com/resource/4334-bgaj.csv") # Download private dataset socrataEmail <- Sys.getenv("SOCRATA_EMAIL", "[email protected]") socrataPassword <- Sys.getenv("SOCRATA_PASSWORD", "7vFDsGFDUG") privateResourceToReadCsvUrl <- "https://soda.demo.socrata.com/resource/a9g2-feh2.csv" # dataset read.socrata(url = privateResourceToReadCsvUrl, email = socrataEmail, password = socrataPassword) # Using an API key to read datasets (reduces throttling) token <- "ew2rEMuESuzWPqMkyPfOSGJgE" df <- read.socrata("https://soda.demo.socrata.com/resource/4334-bgaj.csv", app_token = token) nrow(df) closeAllConnections() ## End(Not run)## Not run: # Human-readable URL: url <- "https://soda.demo.socrata.com/dataset/USGS-Earthquakes-for-2012-11-01-API/4334-bgaj" df <- read.socrata(url) # SoDA URL: df <- read.socrata("https://soda.demo.socrata.com/resource/4334-bgaj.csv") # Download private dataset socrataEmail <- Sys.getenv("SOCRATA_EMAIL", "[email protected]") socrataPassword <- Sys.getenv("SOCRATA_PASSWORD", "7vFDsGFDUG") privateResourceToReadCsvUrl <- "https://soda.demo.socrata.com/resource/a9g2-feh2.csv" # dataset read.socrata(url = privateResourceToReadCsvUrl, email = socrataEmail, password = socrataPassword) # Using an API key to read datasets (reduces throttling) token <- "ew2rEMuESuzWPqMkyPfOSGJgE" df <- read.socrata("https://soda.demo.socrata.com/resource/4334-bgaj.csv", app_token = token) nrow(df) closeAllConnections() ## End(Not run)
Will convert a human-readable URL to a valid REST API call supported by Socrata. It will accept a valid API URL if provided by users and will also convert a human-readable URL to a valid API URL. Will accept queries with optional API token as a separate argument or will also accept API token in the URL query. Will resolve conflicting API token by deferring to original URL.
validateUrl(url, app_token)validateUrl(url, app_token)
url |
- a string; character vector of length one |
app_token |
- a string; SODA API token used to query the data portal https://dev.socrata.com/consumers/getting-started.html |
a - valid Url
Tom Schenk Jr [email protected]
Method for updating Socrata datasets
write.socrata( dataframe, dataset_json_endpoint, update_mode, email, password, app_token = NULL )write.socrata( dataframe, dataset_json_endpoint, update_mode, email, password, app_token = NULL )
dataframe |
- dataframe to upload to Socrata |
dataset_json_endpoint |
- Socrata Open Data Application Program Interface (SODA) endpoint (JSON only for now) |
update_mode |
- "UPSERT" or "REPLACE"; consult https://dev.socrata.com/publishers/getting-started.html |
email |
- The email to the Socrata account with read access to the dataset |
password |
- The password associated with the email to the Socrata account |
app_token |
- a (non-required) string; SODA API token can be used to query the data portal https://dev.socrata.com/consumers/getting-started.html |
Mark Silverberg [email protected]
## Not run: # Store user email and password socrataEmail <- Sys.getenv("SOCRATA_EMAIL", "[email protected]") socrataPassword <- Sys.getenv("SOCRATA_PASSWORD", "7vFDsGFDUG") datasetToAddToUrl <- "https://soda.demo.socrata.com/resource/xh6g-yugi.json" # dataset # Generate some data x <- sample(-1000:1000, 1) y <- sample(-1000:1000, 1) df_in <- data.frame(x,y) # Upload to Socrata write.socrata(df_in,datasetToAddToUrl,"UPSERT",socrataEmail,socrataPassword) ## End(Not run)## Not run: # Store user email and password socrataEmail <- Sys.getenv("SOCRATA_EMAIL", "[email protected]") socrataPassword <- Sys.getenv("SOCRATA_PASSWORD", "7vFDsGFDUG") datasetToAddToUrl <- "https://soda.demo.socrata.com/resource/xh6g-yugi.json" # dataset # Generate some data x <- sample(-1000:1000, 1) y <- sample(-1000:1000, 1) df_in <- data.frame(x,y) # Upload to Socrata write.socrata(df_in,datasetToAddToUrl,"UPSERT",socrataEmail,socrataPassword) ## End(Not run)