ABSTRACT:

Anticipated changes to climate, human population, land use, and urban form will alter the hydrology and availability of water within the water systems on which the world’s population relies. Understanding the effects of these changes will be paramount in sustainably managing water resources, as well as maintaining associated capacity to provide ecosystem services (e.g., regulating flooding, maintaining instream flow during dry periods, cycling nutrients, and maintaining water quality). It will require better information characterizing both natural and human mediated hydrologic systems and enhanced ability to generate, manage, store, analyze, and share growing volumes of observational data. Over the past several years, a number of hydrology domain cyberinfrastructures have emerged or are currently under development that are focused on providing integrated access to and analysis of data for cross-domain synthesis studies. These include the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS), the Critical Zone Observatory Information System (CZOData), HydroShare, the BiG CZ software system, and others. These systems have focused on sharing, integrating, and analyzing hydrologic observations data. This presentation will describe commonalities and differences in the cyberinfrastructure approaches used by these projects and will highlight successes and lessons learned in addressing the challenges of big and complex data. It will also identify new challenges and opportunities for next generation cyberinfrastructure and a next generation of cyber-savvy scientists and engineers as developers and users.

ABSTRACT:

How do you manage, track, and share hydrologic data and models within your research group? Do you find it difficult to keep track of who has access to which data and who has the most recent version of a dataset or research product? Do you sometimes find it difficult to share data and models and collaborate with colleagues outside your home institution? Would it be easier if you had a simple way to share and collaborate around hydrologic datasets and models? HydroShare is a new, web-based system for sharing hydrologic data and models with specific functionality aimed at making collaboration easier. Within HydroShare, we have developed new functionality for creating datasets, describing them with metadata, and sharing them with collaborators. In HydroShare we cast hydrologic datasets and models as “social objects” that can be published, collaborated around, annotated, discovered, and accessed. In this presentation, we will discuss and demonstrate the collaborative and social features of HydroShare and how it can enable new, collaborative workflows for you, your research group, and your collaborators across institutions. HydroShare’s access control and sharing functionality enable both public and private sharing with individual users and collaborative user groups, giving you flexibility over who can access data and at what point in the research process. HydroShare can make it easier for collaborators to iterate on shared datasets and models, creating multiple versions along the way, and publishing them with a permanent landing page, metadata description, and citable Digital Object Identifier (DOI). Functionality for creating and sharing resources within collaborative groups can also make it easier to overcome barriers such as institutional firewalls that can make collaboration around large datasets difficult. Functionality for commenting on and rating resources supports community collaboration and quality evaluation of resources in HydroShare.

This presentation was delivered as part of a Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Cyberseminar in June 2016. Cyberseminars are recorded, and archived recordings are available via the CUAHSI website at http://www.cuahsi.org.

ABSTRACT:

This resource contains the final data files and R scripts used in our analysis of water use across two high-traffic, public restrooms on Utah State University's campus. We used an inexpensive, open source, water metering system that uses off-the-shelf electronic components and inexpensive analog meters to measure water use quantity and behavior at high temporal frequency (< 5 s). We demonstrated this technology in the two restrooms at Utah State University before and after installing high efficiency, automatic faucets and toilet flush valves. We also integrated an inexpensive sensor to count user traffic to the restrooms. Sensing and recording restroom visits and water use events at high frequency allowed us to monitor water use behavior and identify water fixture malfunctions, such as undesired leaks. Results also show average water use per person, variability in water use by different fixtures (faucets versus urinals and toilets), variability in water use by fixtures compared to manufacturer specifications, gender differences in water use, and the difference in water use related to retrofit of the restrooms with high efficiency fixtures. The inexpensive metering system can help institutions remotely measure and record water use trends and behavior, identify leaks and fixture malfunctions, and schedule fixture maintenance or upgrades based on their operation, all of which can ultimately help them meet goals for sustainable water use.

ABSTRACT:

This resource contains stage data collected at multiple sites in the South Fork of the Little Bear River in Cache County, UT. These data resulted from a project funded by the National Science Foundation originally aimed at testing sensor networks and environmental cyberinfrastructure within a hydrologic observatory test bed. The time series of continuous stage measurements and manually read stage plate readings are contained in the files labeled "Stage". Manual discharge measurements, surveyed water surface elevations, and corresponding quality controlled stage values are contained in the files labeled "StageDischarge". These values can be used to derive a stage-discharge relationship for estimating discharge at each site. Notes regarding quality control are provided in each file.

ABSTRACT:

This resource provides three examples of how to use ODM2 and its Python application programming interface (API). The three examples are as follows:

1. ODM2_Example1.ipynb: Create a blank ODM2 database and load controlled vocabulary terms from http://vocabulary.odm2.org in preparation for data loading.
2. ODM2_Example2.ipynb: Parse water quality sample data from an ODM2 Excel Template file using the ODM2 Python API and write data to an ODM2 database instance and a YAML Observations Data Archive (YODA) file.
3. ODM2_Example3.ipynb: Read water quality sample data from an ODM2 database instance using the ODM2 Python API for manipulation or visualization of the data.

Instructions for running each of these Notebooks using HydroShare's JupyterHub server are in the "readme.pdf" file in the Content section below.

The data used by these examples are packaged in the "data" folder within the content section below. Code for generating the ODM2 schema, loading the controlled vocabularies, the ODM2 Python API, and the YODA Tools utilities has been packaged for convenience in the "code" folder in the content section below. Each of these are available in their respective GitHub repositories see http://github.com/ODM2/.

ABSTRACT:

This presentation describes Version 2 of the Observations Data Model (ODM2). ODM2 is an information model for describing and encoding spatially-discrete Earth observations. ODM2 and its related software ecosystem follow an open development model and can be found in GitHub at http://github.com/ODM2.

ABSTRACT:

This dataset includes data collected using a mobile sensing platform during baseflow and stormflow conditions in the Northwest Field Canal, located in Logan, UT. Data were collected by floating a payload of sensors in a longitudinal transect down the length of the canal and recording latitude, longitude, and several water quality variables, including fluorescent dissolved organic matter (FDOM), observations from custom fluorometers designed for calculating the fluorescence index (FI), dissolved oxygen, temperature, pH, specific conductance, and turbidity. The methods used in collection and processing of these data are described in detail in the methods document included within this resource.

ABSTRACT:

This dataset includes grab sample data collected during baseflow and stormflow conditions in the Northwest Field Canal (NWFC), located in Logan, UT. Grab sample data includes results from samples that were analyzed using dissolved organic carbon concentration analysis and excitation emission matrix spectroscopy to determine organic matter concentration and characteristics. Methods used in sample collection and analysis are described in detail within the methods document included as part of this resource.

ABSTRACT:

This dataset includes time series data collected during baseflow and stormflow conditions in the Northwest Field Canal, located in Logan, UT. Time series data includes fluorescent dissolved organic matter (FDOM), observations from custom fluorometers used to calculate the fluorescence index in situ, turbidity, and rainfall. Methods used for deploying sensors, collecting data, and processing for quality control are described in the methods document contained within this resource.

ABSTRACT:

There are now many ongoing efforts to develop low-cost, open-source, low-power sensors and datalogging solutions for environmental applications. Many of these have advanced to the point that high quality scientific measurements can be made using relatively inexpensive and increasingly off-the-shelf components. With the development of these innovative systems, however, comes the ability to generate large volumes of high-frequency monitoring data and the challenge of how to log, transmit, store, and share the resulting data. This presentation will focus on a new, web-based system http://data.envirodiy.org that was designed to enable citizen scientists to stream sensor data from a network of EnviroDIY Mayfly Arduino-based dataloggers. This system enables registration of new sensor nodes through a website. Once registered, any Internet connected device (e.g., cellular or WIFI) can then post data to the data.envirodiy.org website through a web service programming interface. Data are stored in a back-end data store that implements Version 2 of the Observations Data Model (ODM2). Live data can then be viewed and downloaded from the data.envirodiy.org website in a simple text format. While this system was purpose built to support an emerging network of Arduino-based sensor nodes deployed by citizen scientists in the Delaware River Basin, the architecture and components are generic and could be used by any Internet connected device capable of making measurements and formulating an HTTP POST request to send them to data.envirodiy.org.

ABSTRACT:

This resource includes water quality samples collected within the Northwest Field Canal, located in Logan, UT and from stormwater outfalls that drain to the canal. These data were collected to with the purpose of developing surrogate relationships between in situ parameters and total suspended solids (TSS) and to determine spatial loading patterns in the drainage. Samples were collected manually and via an ISCO 3700 automated sampler. These samples were then analyzed for TSS), total phosphorus (TP), and total dissolved phosphorus (TDP). Methods implemented for sample collection and analysis are described within the resource.

ABSTRACT:

This resource includes a data file containing high resolution water use data from a residential dormitory building on Utah State University's campus. The included iPython notebook demonstrates an analysis aimed at answering the question of how weekend (Saturday and Sunday) water demands are different than weekday (Monday - Friday) demands. The code in the Notbook does the following:

1. Resamples the data to hourly by summing incremental volumes
2. Bins the hourly data into weekday versus weekend data
3. Aggregates the data to average hourly and standard deviation by using the "groupby" function
4. Plots the aggregated weekday versus weekend data for visual comparison

ABSTRACT:

HydroShare is a web-based hydrologic information system operated by the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI). Within HydroShare, users can create and share data and models using a variety of file formats and flexible metadata. HydroShare enables users to formally publish these resources as well as create linkages between published data and model resources and peer reviewed journal publications that describe them. Ability to link published data and models with the papers that describe them is a great step in the direction of scientific reproducibility, but is only a first step. HydroShare supports further transparency in the scientific process by enabling scripting of analytical steps via a RESTful application programming interface (API). Using this API, HydroShare users can develop scripts to read data from HydroShare, perform an analytical step (e.g., data processing or visualization), and then write results back to HydroShare. The script itself can then be shared as part of the published dataset in HydroShare, or it can be shared as a Jupyter Notebook that can be executed within the HydroShare environment. Scripts or Jupyter Notebooks can then be executed by others to reproduce the analysis used by the original authors. In this presentation, we discuss how HydroShare can enable best practices for linking publications with data and models and for promoting reproducibility in environmental analyses through sharing of data, models, and scripts that encode the scientific workflow. The HydroShare system is available at http://www.hydroshare.org. Source code for HydroShare is available at https://github.com/hydroshare.

ABSTRACT:

This dataset includes time series data collected at two sites during baseflow and storm event conditions in the Northwest Field Canal and six sites in stormwater outfalls that drain to the Northwest Field Canal, in Logan UT. Time series data includes gage height, pH, dissolved oxygen, water temperature, specific conductance, turbidity, precipitation, and derived discharge observations at the two sites within the canal. Additionally, water temperature, velocity, water depth, discharge, and precipitation observations are included that were made at the stormwater outfall sites. These data were collected to enable the development of surrogate relationships and high frequency estimates of phosphorus and total suspended solids concentrations within the Northwest Field Canal/Logan City water system. Methods used for deploying sensors, collecting data, and processing for quality control are described in the methods document contained within this resource.

ABSTRACT:

This resource contains an analysis of water use within a residential building on Utah State University's campus in Logan, UT. It is also an example of how a data file can be shared in HydroShare along with a Jupyter Notebook for executing code to achieve a reproducible result. The data in this resource were collected on the main water meter of a student residential building on Utah State University's campus in Logan, UT. Flow through the meter was recorded every 1 second for a period of nearly a month. The Jupyter Notebook in this resource demonstrates how to open the file, subset and aggregate the data, and then generate a visualization of water use for weekend periods versus weekday periods to demonstrate differences in the timing of residents' water use during those periods.

ABSTRACT:

Hurricane Maria is an example of a natural disaster that caused disruptions to infrastructure resulting in concerns with water treatment failures and potential contamination of drinking water supplies. This dataset is focused on the water quality data collected in Puerto Rico after Hurricane Maria and is part of the larger collaborative RAPID Hurricane Maria project.

This resource consists of Excel workbooks and a SQLite database. Both were populated with data and metadata corresponding to discrete water quality analysis of drinking water systems in Puerto Rico impacted by Hurricane Maria collected as part of the RAPID Maria project. Sampling and analysis was performed by a team from Virginia Tech in February-April 2018. Discrete samples were collected and returned to the lab for ICPMS analysis. Sampling was also conducted in the field for temperature, pH, free and total chlorine, turbidity, and dissolved oxygen. Complete method and variable descriptions are contained in the workbooks and database. There are two separate workbooks: one for ICPMS data and one for field data. All results are contained in the single database. Sites were sampled corresponding to several water distribution systems and source streams in southwestern Puerto Rico. Coordinates are included for the stream sites, but to preserve the security of the water distribution sites, the locations are only identified as within Puerto Rico.

The workbooks follow the specifications for YAML Observations Data Archive (YODA) exchange format (https://github.com/ODM2/YODA-File). The workbooks are templates with sheets containing tables that are mapped to entities in the Observations Data Model 2 (ODM2 - https://github.com/ODM2). Each sheet in the workbook contains directions for its completion and brief descriptions of the attributes. The data in the sheets was converted to an SQLite database following the ODM2 schema that is also contained in this resource. Conversion was performed using a prototype Python translation software (https://github.com/ODM2/YODA-Tools).

ABSTRACT:

This resource is a demo of HydroShare's composite resource functionality. It contains an example of all of HydroShare's content aggregation types.

ABSTRACT:

CUAHSI’s Water Data Services are community developed, open access, and available to everyone. Workshops are used to share and learn how these services can help researchers and teams on a variety of research tasks. We include an overview of how to develop data management plans, which are increasingly required by most funders. Materials describe how to discover and find a broad array of water data-time series, samples, spatial coverages, published datasets, and case study workflows. CUAHSI apps and tools are introduced for expediting and documenting workflows. We have provided interactive curriculum and tutorials with examples of how toShare your data within a group and publish your data with a DOI. Future training opportunities and funding opportunities for graduate students are listed.

This workshop was a featured event at the 2019 UCOWR Annual Water Resources Conference, Tuesday, June 11 from 1:00 p.m. – 3:50 p.m., White Pine Meeting Room, Cliff Lodge Snowbird, Utah

ABSTRACT:

We present an inexpensive, open source, water metering system for measuring water use quantity and behavior at high temporal frequency. We have demonstrated this technology in multiple water metering case studies, including observing water use within two high-traffic, public restrooms at Utah State University (USU) before and after installing high efficiency, automatic faucets and toilet flush valves. For this case study, we also integrated an inexpensive sensor to count user traffic. Sensing restroom visits and water use events allowed us to identify fixture malfunctions, average water use per person, variability in use by fixtures (faucets versus urinals and toilets), variability in use by fixtures compared to manufacturer specifications, gender differences in use, and the difference in use after retrofit of the restrooms with high efficiency fixtures. Additional case study applications to which we have applied this system include investigating differences in water use of residential populations on USU’s campus with varying sociodemographics, investigating the effectiveness of dual flush toilets, and observing water use in residential homes. In this presentation, we describe both the inexpensive hardware we have used for collecting data along with results for each of our case study applications. Inexpensive metering systems like the one we have demonstrated can help institutions remotely measure and record water use trends and behavior, identify leaks and fixture malfunctions, and schedule fixture maintenance or upgrades, all of which can ultimately help them meet goals for sustainable water use.

ABSTRACT:

This is an example resource in HydroShare for demonstrating the Jupyter Notebook functionality.

ABSTRACT:

The files provided here are the supporting data and code files for the analyses presented in "A low-cost, open source monitoring system for collecting high-resolution water use data on magnetically-driven residential water meters," an article in Sensors (https://doi.org/10.3390/s20133655). The data included in this resource were collected in laboratory testing and field deployment of the Cyberinfrastructure for Intelligent Water Supply (CIWS) datalogger, an open source, low cost device capable of collecting high temporal resolution data on magnetically driven water meters. The code included allows replication of the analyses presented in the article, and the raw data included allow for extension of the analyses conducted. In the article we present a low-cost (≈ $150) monitoring system for collecting high resolution residential water use data without disrupting the operation of commonly available water meters. This system was designed for installation on top of analog, magnetically-driven, positive displacement, residential water meters and can collect data at variable time resolution intervals. The system couples an Arduino Pro microcontroller board, a datalogging shield customized for this specific application, and a magnetometer sensor. The system was developed and calibrated at the Utah Water Research Laboratory and was deployed for testing on five single family residences in Logan and Providence, Utah for a period of over 1 month. Battery life for the device was estimated to be over 5 weeks with continuous data collection at a 4 second time interval. Data collected using this system, under ideal installation conditions, was within 2% of the volume recorded by the register of the meter on which they were installed. Results from field deployments are presented to demonstrate the accuracy, functionality, and applicability of the system. Results indicate the device is capable of collecting data at a resolution sufficient for identifying individual water use events and analyzing water use at coarser temporal resolutions. This system is of special interest for water end-use studies, future projections of residential water use, water infrastructure design, and for advancing our understanding of water use timing and behavior. The system’s hardware design and software are open source, are available for potential reuse, and can be customized for specific research needs.

ABSTRACT:

As global populations continue to increase and become more urbanized, relationships between water and energy are becoming more important. Both are limited in supply, but both are required to satisfy the needs of residential water users. In the context of urbanization and residential water use, domestic hot water (DHW), which is a resource consumed in nearly every residential structure in the developed world, represents one of the most significant water-related uses of energy. However, quantifying hot water use and the energy associated with heating it can be difficult. Water and energy use are typically evaluated separately, and paired datasets that enable direct evaluation of hot water use and its associated energy consumption are rare. Yet, quantifying water and water-related energy use are important in better understanding how they are linked and in identifying opportunities for conservation. We collected high resolution water use and water temperature data within five multi-unit residential structures on a college campus and then developed a water and energy budget model for quantifying water and water-related energy consumption within each building. Results showed varying behavioral consumption patterns across the buildings. Results also showed tradeoffs between data volume and ability to quantify use associated with sampling and data recording frequency. This resource is the result of an effort to establish reproducibility of the methods undertaken to quantify and characterize water and water-related energy with high-resolution smart meter data in multi-unit residential structures. This undertaking was a part of the research obligations associated with a Masters thesis completed at Utah State University in Aug 2020.

ABSTRACT:

This resource contains high resolution residential water use data and classified end uses of water for 31 residential homes located in Logan City and Providence City in Cache County, Utah, USA. Data were collected using a low-cost, open source monitoring device that was designed to operate on magnetically driven residential water meters (see https://doi.org/10.3390/s20133655). Data were recorded with a temporal frequency of 4 seconds and were collected for a period of at least two weeks during the summer when outdoor water use was active and two weeks during the winter when no outdoor water use was expected. The event disaggregation and classification was conducted using the tools available in the HydroShare resource at https://doi.org/10.4211/hs.3143b3b1bdff48e0aaebcb4aedf02feb. The data were measured on the meter located on the water supply line to each home and represent a trace of the total water use for each residence. The dataset also includes secondary data about each of the residences at which data were collected. These data have been anonymized to remove any personally identifiable information from participants in this data collection effort.

ABSTRACT:

This resource contains a set of Jupyter Notebooks that provide Python code examples for using the HydroShare Python Client library (hsclient). The hsclient library enables users to automate most of the functions available via HydroShare's web user interface through Python coding. It enables creation of new resources and editing of existing resources. Edits may include changes to metadata elements and/or content files within resources. A link to the GitHub source code repository for hsclient is provided in the related resources section below.

ABSTRACT:

The files provided here are the supporting data and code files for the analyses presented in "An open source cyberinfrastructure for collecting, processing, storing and accessing high temporal resolution residential water use data," an article in Environmental Modelling and Software (https://doi.org/10.1016/j.envsoft.2021.105137). The data included in this resource were processed using the Cyberinfrastructure for Intelligent Water Supply (CIWS) (https://github.com/UCHIC/CIWS-Server), and collected using the CIWS-Node (https://github.com/UCHIC/CIWS-WM-Node) data logging device. CIWS is an open-source, modular, generalized architecture designed to automate the process from data collection to analysis and presentation of high temporal residential water use data. The CIWS-Node is a low cost device capable of collecting this type of data on magnetically driven water meters. The code included allows replication of the analyses presented in the journal paper, and the raw data included allow for extension of the analyses conducted. The journal paper presents the architecture design and a prototype implementation for CIWS that was built using existing open-source technologies, including smart meters, databases, and services. Two case studies were selected to test functionalities of CIWS, including push and pull data models within single family and multi-unit residential contexts, respectively. CIWS was tested for scalability and performance within our design constraints and proved to be effective within both case studies. All CIWS elements and the case study data described are freely available for re-use.

ABSTRACT:

This presentation was delivered to the National Academies of Sciences Engineering and Medicine workshop on Advancing a Systems Approach to Studying the Earth: A Strategy for the National Science Foundation. The workshop was held virtually on February 4, 2021.

ABSTRACT:

The files provided here are the supporting data and code files for the analyses presented in "Residential Water Meters as Edge Computing Nodes: Disaggregating End Uses and Creating Actionable Information ath the Edge," an article submitted to the Sensors journal. The data included in this resource were collected in a field deployment of the Cyberinfrastructure for Intelligent Water Supply (CIWS) Computational Node, an open source, low cost device capable of collecting, processing, and transferring high temporal resolution data on magnetically driven water meters. The code included allows replication of the findings presented in Sections 4.3 and 4.4 of the article, and the raw and processed data included allow for extension of the analyses conducted.

ABSTRACT:

This resource contains a set of Jupyter Notebooks that provide Python code examples for using the Python dataretrieval package for retrieving data from the United States Geological Survey's (USGS) National Water Information System (NWIS).The dataretrieval package is a Python alternative to USGS-R's dataRetrieval package for the R Statistical Computing Environment used for obtaining USGS or Environmental Protection Agency (EPA) water quality data, streamflow data, and metadata directly from web services. The dataretrieval Python package is an alternative to the R package, not a port, in that it reproduces the functionality of the R package but its organization and functionality differ to some degree. The dataretrieval package was originally created by Timothy Hodson at USGS. Additional contributions to the Python package and these Jupyter Notebook examples were created at Utah State University under funding from the National Science Foundation. A link to the GitHub source code repository for the dataretrieval package is provided in the related resources section below.

ABSTRACT:

This resource contains standardized monthly water use data for single family residences (SFR) in the cities of Logan (11/2014 - 11/2018) and Providence (10/2017 - 05/2020), Utah, USA. Meter readings by Logan and Providence city are conducted on different days of the month, depending on the utility’s working schedule. Thus, the volume of water used within a given month must be estimated from two meter readings. We calculated standardized monthly water use, i.e., from the first to the last day of each month, as follows: Vn = DnMR1 * VMR1 / DMR1 + DnMR2 * V_MR2 / D_MR2 , where, Vn is the volume of water used for a month n. VMR1 is the water volume from the first meter reading (MR1) that contains water use for month n. DMR1 is the number of days covered by MR1 (i.e., the number of days since the previous meter reading), and DnMR1 is the number of days within month n to which MR1 applies. VMR2, DMR2, and DnMR2 have the same information for the second meter reading (MR2) that contains water use for month n.

ABSTRACT:

The files provided here are the supporting data and code files for the analyses presented in "Variability in Consumption and End Uses of Water for Residential Users in Logan and Providence, Utah, USA", an article published in JWRPM (https://ascelibrary.org/journal/jwrmd5). The journal paper assessed how differences water consumption are reflected in terms of timing and distribution of end uses across residential properties. The article provides insights into the variability of indoor and outdoor residential water use at the household level from the analysis of four to 23 weeks of 4-second resolution water use data at 31 single family residential properties. The data were collected in the cities of Logan and Providence, Utah, USA between 2019 and 2021. The 4-second resolution data is publicly available on: http://www.hydroshare.org/resource/0b72cddfc51c45b188e0e6cd8927227e. Standardized monthly values for single family residents in both cities were used in the article and are publicly available on: http://www.hydroshare.org/resource/16c2d60eb6c34d6b95e5d4dbbb4653ef. The code and data included in this resource allows replication of the analyses presented in the journal paper, and the raw data included allow for extension of the analyses conducted.

ABSTRACT:

Scientific and related management challenges in the water domain require synthesis of data from multiple domains. Many data analysis tasks are difficult because datasets are large and complex; standard formats for data types are not always agreed upon nor mapped to an efficient structure for analysis; water scientists may lack training in methods needed to efficiently tackle large and complex datasets; and available tools can make it difficult to share, collaborate around, and reproduce scientific work. Overcoming these barriers to accessing, organizing, and preparing datasets for analyses will be an enabler for transforming scientific inquiries. Building on the HydroShare repository’s established cyberinfrastructure, we have advanced two packages for the Python language that make data loading, organization, and curation for analysis easier, reducing time spent in choosing appropriate data structures and writing code to ingest data. These packages enable automated retrieval of data from HydroShare and the USGS’s National Water Information System (NWIS), loading of data into performant structures keyed to specific scientific data types and that integrate with existing visualization, analysis, and data science capabilities available in Python, and then writing analysis results back to HydroShare for sharing and eventual publication. These capabilities reduce the technical burden for scientists associated with creating a computational environment for executing analyses by installing and maintaining the packages within CUAHSI’s HydroShare-linked JupyterHub server. HydroShare users can leverage these tools to build, share, and publish more reproducible scientific workflows. The HydroShare Python Client and USGS NWIS Data Retrieval packages can be installed within a Python environment on any computer running Microsoft Windows, Apple MacOS, or Linux from the Python Package Index using the PIP utility. They can also be used online via the CUAHSI JupyterHub server (https://jupyterhub.cuahsi.org/) or other Python notebook environments like Google Collaboratory (https://colab.research.google.com/). Source code, documentation, and examples for the software are freely available in GitHub at https://github.com/hydroshare/hsclient/ and https://github.com/USGS-python/dataretrieval.

This presentation was delivered as part of the Hawai'i Data Science Institute's regular seminar series: https://datascience.hawaii.edu/event/data-science-and-analytics-for-water/

ABSTRACT:

HydroShare is a web-based repository and hydrologic information system operated by the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) that enables users to share, collaborate around, and publish data, models, code, and applications associated with water related research. This seminar will focus on the capabilities of the HydroShare repository and functionality related to submitting, sharing, and publishing data and research products. It will cover HydroShare’s resource data model, describing HydroShare resources with metadata, and some best practices for depositing data and research products in HydroShare. It will also cover how information technology and best practices can enhance the transparency, reproducibility, and trust in the findings of water-related research by making hydrologic information more findable, accessible, interoperable and reusable (FAIR), and through linked computational systems simplifying the workflows needed for hydrologic modeling and analysis.

This presentation was delivered on December 7, 2021 at the USAID Center for Excellence for Water at Alexandria University Webinar Series.

ABSTRACT:

The purpose of this resource is to automate extraction of discharge data from the United States Geological Survey (USGS) National Water Information System (NWIS) (https://waterdata.usgs.gov/nwis), precipitation from PRISM Climate Group database (https://prism.oregonstate.edu/), and evapotranspiration from the OpenET databases (https://openetdata.org/). This resource is part of a semester project for the USU CEE 6110: Hydroinformatics class (Spring 2022).

ABSTRACT:

The purpose of this HydroShare resource is to facilitate the extraction of monthly-averaged Evapotranspiration (ET) data from the OpenET database (https://openetdata.org/). This resource could be used to extract ET data at one point using its latitude & longitude. The resource could also be used to have an average ET value at the watershed scale using a shapefile of the watershed of interest.
The OpenET uses the best available science to provide easily accessible satellite-based estimates of evapotranspiration (ET) (https://openetdata.org/about/). The OpenET database provides ET data using the Ensemble method.

ABSTRACT:

The files provided here are the supporting data and code files for the analyses presented in "Impact of data temporal resolution on quantifying residential end uses of water", an article submitted to the Water journal (https://www.mdpi.com/journal/water). The journal paper assessed how the temporal resolution at which water use data are collected impacts our ability to identify water end use events, calculate features of individual events, and classify events by end use. Additionally, we also explored implications for data management associated with collecting this type of data as well as methods and tools for analyzing and extracting information from it. The data were collected in the cities of Logan and Providence, Utah, USA in 2022 and are included in this resource. The code and data included in this resource allow replication of the analyses presented in the journal paper, and the raw data included allow for extension of the analyses conducted.

ABSTRACT:

This resource contains raw data collected for the project "Increasing the Water Conservation Impact of Utah State University’s (USU) Extension Water Check Program with 5 Second Metering" (https://uwrl.usu.edu/water-check-study). The data is for ~ 78 households in Logan and Hyde Park, Utah collected in Summer and Fall 2022. 5-second water use data was collected over the entire period using a Flume Smart Home Water Monitoring Device. After ~ two weeks, a USU Extension Water Check was conducted during a site visit. There are 6 data sets in this resource. Data are anonymized and can be linked -- joined -- by the SiteID field.

1_Database_CSVFiles/
1) FlumePropertyData.csv => Metadata for the households collected by Flume when a device is installed and the Flume phone App was installed.
2) Sites.csv => Metadata for the households including city, state, and zipcode.
3) WaterCheckData.csv => Parcel, landscape, and irrigation system data collected as part of the USU Extension Water Check during a 1-hour visit to the household. Data also include Water Check recommendations to reduce irrigation water use.
4) RawWaterUseData/SITE_XXX.csv => Raw 5-second water use data collected by Flume Smart Home Water Monitoring Devices (http:/FlumeWater.com). One file for each household/SiteID. XXX is the SiteID.
5) daily_WeatherData_GVFarm.csv => Weather data from the nearest station - Greenville Farm, Cache Valley, Utah.
6) TrainingData.csv => Irrigation events identified by duration (minutes), volume_gal (gallons), average_fr_GPM (gallons per minute), label (type of event). These data are used to train a model that uses the raw 5-second data to classify irrigation events.
The code to classify the raw 5-second water use data is in a separate code repository - https://github.com/cjbas22/HelpUSUExtensionP.

2_AdditionalData => Folder with duplicate copies of the weather station and training data.

3_Database => Empty folder. Code in the repository https://github.com/cjbas22/HelpUSUExtensionP reads the raw csv files and creates a database with tables for each data file.

ABSTRACT:

The purpose of this resource is to assess conditions in the Logan River at the Main Street site to determine if the
environment is suitable for trout. As a fly fisherman, I am seeking this information to determine if this site will hold
fish and to determine if it is ethical to fish here. The variables to be evaluated are pH levels and water temperature.
pH level has great influence in a trout’s life such as playing a significant role in egg development, size and growth,
and whether or not the trout is able to live in a specific environment. pH levels ranging from 7.1 to 9.0 are the best
conditions for trout (Allen). The optimal feeding and movement water temperature for trout is between 6 and 20
degrees Celsius. When the water temperature rises above 20 degrees Celsius trout begin to become stressed and
water temperatures above 24 degrees Celsius for an extended period can be lethal to trout (Rose). In order to
visualize the pH levels and water temperature in the Logan River at Main Street I have created a plot showing these
two variables over the entire year of 2015. The figure created in the Jupyter Notebook enables the viewer to easily visualize the pH level and water temperature in the Logan River at Main Street throughout the entire year of 2015. This figure is important because it shows that the pH level of the water at this site falls within the optimal range for trout environment. It also shows that the water temperature never exceeds 20 degrees Celsius and therefore temperature does not affect a trout’s stress level at this site. The water temperature falls within the optimal range for trout of 6 to 20 degrees Celsius for the majority of the year. The pH levels and water temperature shown by the plot suggest that the Logan River at Main Street is suitable environment for trout. Therefore, this site could potentially be a good spot for fly fishing.

ABSTRACT:

Scientific and management challenges in the water domain require synthesis of diverse data. Many data analysis tasks are difficult because datasets are large and complex; standard data formats are not always agreed upon or mapped to efficient structures for analysis; scientists may lack training for tackling large and complex datasets; and it can be difficult to share, collaborate around, and reproduce scientific work. Overcoming barriers to accessing, organizing, and preparing datasets for analyses can transform the way water scientists work. Building on the HydroShare repository’s cyberinfrastructure, we have advanced two Python packages that make data loading, organization, and curation for analysis easier, reducing time spent in choosing appropriate data structures and writing code to ingest data. These packages enable automated retrieval of data from HydroShare and the USGS’s National Water Information System (NWIS) (i.e., a Python equivalent of USGS’ R dataRetrieval package), loading data into performant structures that integrate with existing visualization, analysis, and data science capabilities available in Python, and writing analysis results back to HydroShare for sharing and publication. While these Python packages can be installed for use within any Python environment, we will demonstrate how the technical burden for scientists associated with creating a computational environment for executing analyses can be reduced and how sharing and reproducibility of analyses can be enhanced through the use of these packages within CUAHSI’s HydroShare-linked JupyterHub server.

This HydroShare resource includes all of the materials presented in a workshop at the 2023 CUAHSI Biennial Colloquium.

ABSTRACT:

Water science and management challenges require synthesis of diverse data. Many data analysis tasks are difficult because data are large or complex; standard formats are not always agreed upon or mapped to efficient structures for analysis; scientists may lack training for tackling large and complex datasets; and it can be difficult to share, collaborate around, and reproduce scientific work. Access to computing for running and sharing data science or modeling workflows and structuring them in a way that they can be reproduced can also be challenging. Overcoming these barriers can transform the way water scientists work. Participants will learn how to use multiple data science tools, including data retrieval packages for easy access to data from the United States Geological Survey’s (USGS) National Water Information System (NWIS) and tools associated with the CUAHSI HydroShare repository and linked JupyterHub environment available to assist scientists in building, sharing, and publishing more reproducible scientific workflows following Findable, Accessible, Interoperable, and Reusable (FAIR) principles. We will demonstrate how the technical burden for scientists associated with creating a computational environment for executing analyses can be reduced and how sharing and reproducibility of analyses can be enhanced through the use of these tools.

This HydroShare resource includes all of the materials presented in a workshop at WaterSciCon24.

ABSTRACT:

The Utah Division of Water Rights (DWRi) is an agency of Utah State Government within the Department of Natural Resources that administers the measurement, appropriation, apportionment, and distribution of the State’s valuable water resources. The extensive data collection DWRi undertakes to enable management of Utah’s water resources and to meet its mandate requires significant information technology (IT) infrastructure. Data collection hardware (e.g., measurement structures and sensors installed in the field), software, and databases are needed to enable collection, management, and use of the data. While DWRi has developed and operates a water use data infrastructure for the state of Utah, some of the systems used by DWRi have become dated and lack necessary flexibility to meet DWRi’s growing data management needs. To address this, DWRi initiated a process to critically examine their existing data infrastructure and identify areas needing modernization.

This report is the result of a "Hydroinformatics and Technology Gap Analysis" conducted for the Utah Division of Water Rights (DWRi) by Utah State University and provides an in-depth examination of DWRi's current data and IT infrastructure, software systems, and processes. While DWRi manages several other important datasets, this gap analysis document is focused on measurement data (e.g., observations of flow recorded at stream diversions), Water Use Program data (water diversion data primarily from public water suppliers), and water distribution accounting (the procedures used by DWRi to ensure that water is distributed to water users by priority of legal water rights). The analysis aimed to identify the disparities between existing capabilities and the organizational goals, industry standards, and improvements needed as identified by DWRi personnel. The focus of the report is not only on data collection and management but also on the underlying software and database systems, as well as the software development processes employed by DWRi for managing water use data within Utah.


To access the report, scroll to the "Content" section below, right click on the PDF document, and select download. You can also double click on the file to download.
To learn more about Utah Division of Water Rights, visit: https://waterrights.utah.gov/
To access the Technology Modernization Roadmap document produced to address this Gap Analysis, scroll to the Related Resources section below and click on the link.

ABSTRACT:

The Utah Division of Water Rights (DWRi) is an agency of Utah State Government within the Department of Natural Resources that administers the measurement, appropriation, apportionment, and distribution of the state’s valuable water resources. DWRi collects and maintains records of water diversion from surface and underground sources, which are mainly included in water distribution systems within the state. The extensive data collection DWRi undertakes to enable management of Utah’s water resources and to meet its mandate requires significant information technology (IT) infrastructure. Data collection hardware (e.g., measurement structures and sensors installed in the field), software, and databases are needed to enable collection, management, and use of the data. While DWRi has developed and operates a water use data infrastructure for the state of Utah, some of the systems used by DWRi have become dated, lack necessary flexibility, and may not scale to meet DWRi’s growing data management needs. Furthermore, a recent Legislative Audit recommended that DWRi work toward use of best practices for data management to improve effectiveness, increase the availability and usability of DWRi’s data, and to more effectively monitor state water. To address these challenges and recommendations, in 2023 DWRi initiated a process to critically examine their existing data infrastructure and identify areas where modernization and enhancements are needed. That initial effort resulted in a "Hydroinformatics and Technology Gap Analysis" conducted by Utah State University that provided an in-depth examination of DWRi's current data and IT infrastructure, software systems, and processes. The Gap Analysis identified disparities between DWRi’s existing systems and capabilities and the organizational goals, industry standards, and improvements needed as identified by USU and DWRi personnel.

This Technology Modernization Roadmap considers the gaps and recommendations developed through the Gap Analysis and lays out a sequenced plan, or roadmap, for software development and other activities that will help DWRi modernize and advance their data collection and management infrastructure to meet existing operational needs, along with evolving data collection and management requirements associated with new programs. This Roadmap lays out the foundations for implementing data management best practices to improve the usability of DWRi’s data, along with the transparency and accountability of DWRi’s operations for effective water management and planning in Utah for decades to come.

To access the Technology Modernization Roadmap document, scroll to the "Content" section below, right click on the PDF document, and select download. You can also double click on the file to download. To access the Hydroinformatics and Technology Gap Analysis document on which this Roadmap document is based, scroll to the “Related Resources” section below and click the link to the gap analysis document.

To learn more about Utah Division of Water Rights, visit: https://waterrights.utah.gov/