stoqs_march2013

Parameter-Parameter

Linear regression Free range Samples No stride

Rotate: Left mouse
Translate: Ctrl + left mouse
Zoom: Alt + left mouse

Sampled Parameter Data Access

Click on Sample locations in time-depth plot to get information here

	Apache Parquet binary column format

http://stoqs.mbari.org:8000/stoqs_march2013/api/measuredparameter.parquet?collect=standard_name

	Keyhole Markup Language - click on icon to view in Google Earth

Select a single sampled parameter for data access to view your selection in Google Earth

	Structured Query Language for PostgreSQL

	Matlab - Load data into Matlab structure array

Install this m-file on your system (instructions are in the comments):
https://raw.githubusercontent.com/stoqs/stoqs/master/stoqs/contrib/analysis/dbconn_jdbc_stoqs.m

Then use the 'SQL' in the adjacent tab to load data into Matlab.

	JavaScript Object Notation

http://stoqs.mbari.org:8000/stoqs_march2013/api/sampledparameter.json

	Comma Separated Values

http://stoqs.mbari.org:8000/stoqs_march2013/api/sampledparameter.csv

	Tabbed Separated Values

http://stoqs.mbari.org:8000/stoqs_march2013/api/sampledparameter.tsv

	Hyper Text Markup Language table

http://stoqs.mbari.org:8000/stoqs_march2013/api/sampledparameter.html

Measured Parameter Data Access

	Apache Parquet binary column format

Collect:	standard_name(s) name(s)
Include:	Activity names

	Estimated	Available
RAM (GB)	...	...
Time (min)	...	...
Rows	...
File size (MB)	...

Click "Estimate requirements" button before using this Data Access URL

http://stoqs.mbari.org:8000/stoqs_march2013/api/measuredparameter.parquet?collect=standard_name

In addition to using the above ad hoc .parquet request generation capability you may also script up requests by activity__name, inherently reducing server-side memory and time requirements. Here is an example of reading Dorado data survey-by-survey into a Matlab session:

% File: read_parquet.m
% Read parquet files from STOQS and report the time it takes
campaign_url = "https://stoqs.mbari.org/stoqs_all_dorado";
api_call = "/api/measuredparameter.parquet?collect=name&activity__name__contains=";
surveys = ["Dorado389_2020_006_06_006_06_decim", "Dorado389_2020_035_00_035_00_decim"];
wo = weboptions('CertificateFilename', '', 'Timeout', 300);
for i=1:length(surveys),
    tic;
    disp(strcat("Reading ", surveys(i), " as .parquet"));
    parquet_url = strcat(campaign_url, api_call, surveys(i));
    file = websave(strcat(surveys(i), '.parquet'), parquet_url, wo);
    disp(strcat("Wrote ", file));
    toc;
    tic;
    info = parquetinfo(file);
    T = parquetread(file);
    disp(strcat("Read ", file));
    toc;
end
disp(info.VariableNames)

—————————————————

>> read_parquet
Reading Dorado389_2020_006_06_006_06_decim as .parquet
Wrote /u/mccann/matlab/Dorado389_2020_006_06_006_06_decim.parquet
Elapsed time is 14.278701 seconds.
Read /u/mccann/matlab/Dorado389_2020_006_06_006_06_decim.parquet
Elapsed time is 0.410685 seconds.
Reading Dorado389_2020_035_00_035_00_decim as .parquet
Wrote /u/mccann/matlab/Dorado389_2020_035_00_035_00_decim.parquet
Elapsed time is 14.070438 seconds.
Read /u/mccann/matlab/Dorado389_2020_035_00_035_00_decim.parquet
Elapsed time is 0.097718 seconds.
    Columns 1 through 11

    "adinos"    "altitude"    "bbp420 (m-1)"    "bbp700 (m-1)"    "bg_biolum (ph L^..."    "biolume"    "diatoms"    "fl700_uncorr"    "fluo"    "hdinos"    "intflash (ph s^{..."

    Columns 12 through 19

    "mepCountList (co..."    "nbflash_high (L^..."    "nbflash_low (L^{..."    "nitrate (micromo..."    "oxygen (ml/l)"    "pitch (degree)"    "profile"    "roll (degree)"

    Columns 20 through 30

    "salinity"    "sepCountList (co..."    "sigmat"    "spice"    "temperature (Cel..."    "yaw (degree)"    "platform"    "timevalue"    "depth"    "latitude"    "longitude"

Here is visualize_parquet.ipynb which shows how to read and visualize parquet data in Python.
Here is parquet2csv.py which you can execute at the command line to convert parquet data to a comma separated value file.

	Keyhole Markup Language - click on icon to view in Google Earth

Select a single measured parameter for data access to view your selection in Google Earth

	Structured Query Language for PostgreSQL

	Matlab - Load data into Matlab structure array

Install this m-file on your system (instructions are in the comments):
https://raw.githubusercontent.com/stoqs/stoqs/master/stoqs/contrib/analysis/dbconn_jdbc_stoqs.m

Then use the 'SQL' in the adjacent tab to load data into Matlab.

	JavaScript Object Notation

http://stoqs.mbari.org:8000/stoqs_march2013/api/measuredparameter.json

	Comma Separated Values

http://stoqs.mbari.org:8000/stoqs_march2013/api/measuredparameter.csv

	Tabbed Separated Values

http://stoqs.mbari.org:8000/stoqs_march2013/api/measuredparameter.tsv

	Hyper Text Markup Language table

http://stoqs.mbari.org:8000/stoqs_march2013/api/measuredparameter.html

Click and drag to zoom

Mar 15

Mar 16

Mar 17

Mar 18

Mar 19

Mar 20

Mar 21

Mar 22

Mar 23

Mar 24

Mar 25

-100

100

200

300

400

500

Time (gmt)

Depth (m)

Click and drag to zoom

Color by: Platform Activity

Measured Parameters

Parameter-Parameter				Plot Data
X	Y	Z	C	Lines	Color
						Clear Selection
						Longitude
						Latitude
						Depth
						Time
							\|\|\|\|\|
							\|\|
							\|\|
							\|
							\|
							\|
							\|\|
							\|
							\|
							\|
							\|\|
							\|
							\|\|
							\|
							\|\|\|
							\|\|\|\|\|
							\|\|\|\|\|
							\|\|
							\|\|\|
							\|
							\|\|
							\|

Parameter Values

Show distribution of parameter values

Attributes

Platforms

	Space	Time
			Check all / Clear

ship

Dorado389_2013_074_02_074_02_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
Dorado389_2013_075_05_075_06_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
Dorado389_2013_076_01_076_02_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
Dorado389_2013_079_04_079_04_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
Dorado389_2013_080_02_080_02_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
Dorado389_2013_081_05_081_05_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
Dorado389_2013_081_06_081_06_decim.nc	2column	biolume	hist_stats	lopc	nav_adjust	prof_stats
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013074/2013.074.02/lopc.nc (stride=1)
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013075/2013.075.05/lopc.nc (stride=1)
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013076/2013.076.01/lopc.nc (stride=1)
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013079/2013.079.04/lopc.nc (stride=1)
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013080/2013.080.02/lopc.nc (stride=1)
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013081/2013.081.05/lopc.nc (stride=1)
http://dods.mbari.org/opendap/data/ssdsdata/ssds/generated/netcdf/files/ssds.shore.mbari.org/auvctd/missionlogs/2013/2013081/2013.081.06/lopc.nc (stride=1)

algae		algae_r
amp		amp_r
balance		balance_r
curl		curl_r
deep		deep_r
delta		delta_r
dense		dense_r
diff		diff_r
gray		gray_r
haline		haline_r
ice		ice_r
matter		matter_r
oxy		oxy_r
phase		phase_r
rain		rain_r
solar		solar_r
speed		speed_r
tarn		tarn_r
tempo		tempo_r
thermal		thermal_r
topo		topo_r
turbid		turbid_r

cividis		cividis_r
inferno		inferno_r
magma		magma_r
plasma		plasma_r
viridis		viridis_r

Blues		Blues_r
BuGn		BuGn_r
BuPu		BuPu_r
GnBu		GnBu_r
Greens		Greens_r
Greys		Greys_r
OrRd		OrRd_r
Oranges		Oranges_r
PuBu		PuBu_r
PuBuGn		PuBuGn_r
PuRd		PuRd_r
Purples		Purples_r
RdPu		RdPu_r
Reds		Reds_r
YlGn		YlGn_r
YlGnBu		YlGnBu_r
YlOrBr		YlOrBr_r
YlOrRd		YlOrRd_r

Wistia		Wistia_r
afmhot		afmhot_r
autumn		autumn_r
binary		binary_r
bone		bone_r
cool		cool_r
copper		copper_r
gist_gray		gist_gray_r
gist_heat		gist_heat_r
gist_yarg		gist_yarg_r
gray		gray_r
hot		hot_r
pink		pink_r
spring		spring_r
summer		summer_r
winter		winter_r

Spatial

Measured Parameter

Platforms

Animation

Simulation

Terrain

Metadata: loading...

RachelCarson_PCTD

RachelCarson_UCTD

daphne

dorado

tethys

RachelCarson_PCTD

RachelCarson_UCTD

daphne

dorado

tethys

Parameter-Parameter

Sampled Parameter Data Access

Measured Parameter Data Access

Temporal

Colormaps

Sampled Parameters

Measured Parameters

Parameter Values

Attributes

Platforms

auv

ship

BrBG		BrBG_r
PRGn		PRGn_r
PiYG		PiYG_r
PuOr		PuOr_r
RdBu		RdBu_r
RdGy		RdGy_r
RdYlBu		RdYlBu_r
RdYlGn		RdYlGn_r
Spectral		Spectral_r
bwr		bwr_r
coolwarm		coolwarm_r
seismic		seismic_r

Accent		Accent_r
Dark2		Dark2_r
Paired		Paired_r
Pastel1		Pastel1_r
Pastel2		Pastel2_r
Set1		Set1_r
Set2		Set2_r
Set3		Set3_r
tab10		tab10_r
tab20		tab20_r
tab20b		tab20b_r
tab20c		tab20c_r

CMRmap		CMRmap_r
brg		brg_r
cubehelix		cubehelix_r
flag		flag_r
gist_earth		gist_earth_r
gist_ncar		gist_ncar_r
gist_rainbow		gist_rainbow_r
gist_stern		gist_stern_r
gnuplot		gnuplot_r
gnuplot2		gnuplot2_r
jet		jet_r
nipy_spectral		nipy_spectral_r
ocean		ocean_r
prism		prism_r
rainbow		rainbow_r
terrain		terrain_r

Restore this database on your system

Share this view

Metadata: loading...

Parameter-Parameter Clear Selection

Temporal Clear Selection

Sampled Parameters Clear Selection

Measured Parameters Clear Selection

Parameter Values Clear Selection

Attributes Clear Selection

Platforms Clear Selection

Parameter-Parameter

Temporal

Sampled Parameters

Measured Parameters

Parameter Values

Attributes

Platforms