An official website of the United States government
This Jupyter notebook demonstrates using the hdpws Python package to access Space Physics Archive Search and Extract (SPASE) metadata documents from the Heliophysics Data Portal (HDP). It assumes some familarity with the SPASE data model. This notebook contains the following sections:
Install the prerequisite software from Python Package Index (PyPI) software repository.
Execute some preliminary code that is necessary before the code that follows.
) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.from hdpws.hdpws import HdpWs
from hdpws import NAMESPACES as NS
from hdpws.resourcetype import ResourceType as rt
from hdpws.spase import AccessURL, HapiAccessURL
from IPython.core.display import HTML
hdp = HdpWs()
# limit display of long lists
DISP_LIMIT = 5
def display_link(
url: str,
text: str) -> None:
display(HTML(f' <a href="{url}" target="_blank">{text}</a>'))
The following code demonstrates how to get the list of available /Spase/Software/ApplicationInterface values.
result = hdp.get_application_interfaces()
print('HDP ApplicationInterfaces:')
for value in result['ApplicationInterface']:
print(f' {value}')
HDP ApplicationInterfaces:
CLI
GUI
API
The following code demonstrates how to get the list of available /Spase/MeasurementType values.
result = hdp.get_measurement_types()
print(f'HDP MeasurementTypes:')
for i, value in enumerate(result['MeasurementType']):
print(f'{i}. {value}')
HDP MeasurementTypes: 0. ActivityIndex 1. Dopplergram 2. Dust 3. ElectricField 4. EnergeticParticles 5. Ephemeris 6. ImageIntensity 7. InstrumentStatus 8. IonComposition 9. Irradiance 10. MagneticField 11. Magnetogram 12. NeutralAtomImages 13. NeutralGas 14. Profile 15. Radiance 16. Spectrum 17. SPICE 18. ThermalPlasma 19. Waves 20. Waves.Active 21. Waves.Passive
The following code demonstrates how to get the list of available /Spase/SpectralRange values.
result = hdp.get_spectral_ranges()
print(f'HDP SpectralRanges:')
for i, value in enumerate(result['SpectralRange']):
print(f'{i}. {value}')
HDP SpectralRanges: 0. CaK 1. ExtremeUltraviolet 2. FarUltraviolet 3. GammaRays 4. Halpha 5. HardXRays 6. He10830 7. He304 8. Infrared 9. K7699 10. LBHBand 11. Microwave 12. NaD 13. Ni6768 14. Optical 15. RadioFrequency 16. SoftXRays 17. Ultraviolet 18. WhiteLight 19. XRays
The following code demonstrates how to get the list of available /Spase/PhenomenonType values.
result = hdp.get_phenomenon_types()
print(f'HDP PhenomenonTypes:')
for i, value in enumerate(result['PhenomenonType']):
print(f'{i}. {value}')
HDP PhenomenonTypes: 0. ActiveRegion 1. Aurora 2. BowShockCrossing 3. CoronalHole 4. CoronalMassEjection 5. EITWave 6. EnergeticSolarParticleEvent 7. ForbushDecrease 8. GeomagneticStorm 9. InterplanetaryCoronalMassEjection 10. InterplanetaryShock 11. MagneticCloud 12. MagnetopauseCrossing 13. RadioBurst 14. SectorBoundaryCrossing 15. SolarFlare 16. SolarWindExtreme 17. StreamInteractionRegion 18. Substorm
The following code demonstrates how to get the list of available /Spase/ObservedRegion values.
result = hdp.get_observed_regions()
observed_regions = result['ObservedRegion']
print(f'HDP Observed Regions:')
for i, value in enumerate(observed_regions):
print(f'{i}. {value}')
if i == DISP_LIMIT:
print('...')
print(f'{len(observed_regions) - 1}. {observed_regions[-1]}')
break
HDP Observed Regions: 0. Asteroid 1. Comet 2. Comet.1PHalley 3. Comet.26PGriggSkjellerup 4. Comet.67PChuryumovGerasimenko 5. Earth ... 124. Venus.Magnetosphere.RingCurrent
The following code demonstrates how to get the list of available /Spase/Observatory/ResourceID values.
result = hdp.get_observatory_ids()
observatory_ids = result['ObservatoryID']
print(f'HDP ObservatoryIDs:')
for i, value in enumerate(observatory_ids):
print(f'{i}. {value}')
if i == DISP_LIMIT:
print('...')
print(f'{len(observatory_ids) - 1}. {observatory_ids[-1]}')
break
HDP ObservatoryIDs: 0. spase://SMWG/Observatory/AE-D 1. spase://SMWG/Observatory/Helios1 2. spase://SMWG/Observatory/DynamicsExplorer1 3. spase://SMWG/Observatory/SolarOrbiter 4. spase://SMWG/Observatory/IMP8 5. spase://SMWG/Observatory/MarsExpress ... 4043. spase://SMWG/Observatory/UNH
The following code demonstrates how to get a list of available //Keyword values.
result = hdp.get_keywords()
keywords = result['Keyword']
print(f'HDP Keywords:')
for i, value in enumerate(keywords):
print(f'{i}. {value}')
if i == DISP_LIMIT:
print('...')
print(f'{len(keywords) - 1}. {keywords[-1]}')
break
HDP Keywords: 0. 60 sec 1. ground magnetometer data 2. electroject ionospheric currents 3. auroral electrojet index 4. 10 sec 5. spherical elementary current amplitudes ... 1518. WDC Station Code: KZN
The following code demonstrates how to get SPASE Numerical/Display data documents matching the specified search criteria.
query = {
'InstrumentID': 'spase://SMWG/Instrument/ACE/CRIS',
'Cadence': '=PT1H',
'ObservedRegion': 'Heliosphere.NearEarth',
'MeasurementType': 'EnergeticParticles',
'AccessRights': 'Open',
'Format': 'CDF'
}
types = [rt.NUMERICAL_DATA, rt.DISPLAY_DATA]
time_range = ['2022-01-01', '2022-01-02']
result = hdp.get_spase_data(types, query, time_range)
if result['HttpStatus'] == 200:
for i, spase in enumerate(result['Result'].findall('.//Spase', namespaces=NS)):
id = spase.find('.//ResourceID', namespaces=NS)
name = spase.find('.//Name', namespaces=NS)
description = spase.find('.//Description', namespaces=NS)
#print(name.text)
print(f'{i}. ResourceID: {id.text}')
print('--------------------------------------------------')
print(f'Description: {description.text[:60]} ...')
display_link(hdp.get_spase_url(id.text), 'HTML representation of SPASE')
ws_access_url_element = \
spase.find('.//AccessURL[Style="WebService"]', namespaces=NS)
if ws_access_url_element is not None:
ws_access_url = AccessURL(ws_access_url_element)
print('Web Service access:')
display_link(ws_access_url.url, 'Example data access code')
#print(' URL: ', ws_access_url.url)
hapi_access_url_element = \
spase.find('.//AccessURL[Style="HAPI"]', namespaces=NS)
if hapi_access_url_element is not None:
hapi_access_url = HapiAccessURL(hapi_access_url_element)
print('HAPI access:')
print(' URL: ', hapi_access_url.url)
print(' Dataset IDs: ', hapi_access_url.product_key)
hapi_display_url = hapi_access_url.get_html_url()
display_link(hapi_access_url.get_html_url(), 'HAPI access information')
print()
0. ResourceID: spase://NASA/NumericalData/ACE/CRIS/L2/PT1H -------------------------------------------------- Description: ACE Cosmic Ray Isotope Spectrometer (CRIS) intensities and c ...
Web Service access:
HAPI access:
URL: https://cdaweb.gsfc.nasa.gov/hapi
Dataset IDs: ['AC_H2_CRIS']
1. ResourceID: spase://NASA/NumericalData/ParkerSolarProbe/SWEAP/SPAN-I/Level2/AlphaDifferentialEnergyFlux/VariableCadence -------------------------------------------------- Description: Parker Solar Probe SPAN-I Alpha particle differential energy ...
Web Service access:
HAPI access:
URL: https://cdaweb.gsfc.nasa.gov/hapi
Dataset IDs: ['PSP_SWP_SPI_SF01_L2_8DX32EX8A']
The following code demonstrates how to get a SPASE Catalog document matching the specified criteria.
query = {
# 'InstrumentID': 'spase://SMWG/Instrument/ACE/MAG',
# 'PhenomenonType': 'MagneticCloud',
# 'Description': 'ICME'
'Keyword': 'current sheet'
}
#time_range = ['1999-01-01', '1999-01-02']
time_range = ['2005-01-01', '2014-01-01']
result = hdp.get_spase_catalog(query, time_range)
if result['HttpStatus'] == 200:
print('Result Catalogs:')
for catalog in result['Result'].findall('.//Catalog',
namespaces=NS):
print('ResourceID: ', catalog.findall('.//ResourceID',
namespaces=NS)[0].text)
print('ResourceName: ', catalog.findall('.//ResourceName',
namespaces=NS)[0].text)
print('Description:',
catalog.findall('.//Description',
namespaces=NS)[0].text[:750],
'...')
Result Catalogs: ResourceID: spase://NASA/Catalog/Eriksson/WindCurrentSheetExhaustList ResourceName: Wind Current Sheet_Exhaust Event List in LMN Coordinates Description: The dataset contains a list of solar wind current sheet exhaust event data derived from Wind interplanetary magnetic field and solar wind plasma observations. The list is described in a peer reviewed article, Eriksson et al. 2022, titled: Characteristics of Multi-Scale Current Sheets in the Solar Wind at 1 AU Associated with Magnetic Reconnection and the Case for a Heliospheric Current Sheet Avalanche. The data is listed as an ASCII table with eleven columns including the current sheet (CS) start date/time and stop date/time, and the three unit vectors of each CS in a GSE base coordinate system. Here, NGSE is the CS normal direction, LGSE is the direction of the reconnecting component of the magnetic field, and MGSE is the cross-product MGS ...
The following code demonstrates how to get a SPASE Collection document matching the specified criteria.
query = {
'ResourceID': 'spase://NASA/Collection/IRIS/IRIS_SDOAIA',
'MemberID': 'spase://NASA/NumericalData/IRIS/IRIS/PT1S',
'Description': 'IRIS AND SDO and AIA'
}
result = hdp.get_spase_collection(query)
if result['HttpStatus'] == 200:
for collection in result['Result'].findall('.//Collection',
namespaces=NS):
print('ResourceID: ', collection.find('.//ResourceID',
namespaces=NS).text)
print('ResourceName: ', collection.find('.//ResourceName',
namespaces=NS).text)
print('Collection members:')
for member in collection.findall('.//Member',
namespaces=NS):
member_id = member.find('.//MemberID',
namespaces=NS)
print(f' {member_id.text}')
else:
print(result['HttpStatus'])
ResourceID: spase://NASA/Collection/IRIS/IRIS_SDOAIA
ResourceName: Co-aligned Interface Region Imaging Spectrograph (IRIS) and Solar Dynamics Observatory (SDO) Atmospheric Imaging Assembly (AIA) Observations
Collection members:
spase://NASA/NumericalData/IRIS/IRIS/PT1S
spase://NASA/NumericalData/SDO/AIA/PT12S
The following code demonstrates how to get the SPASE Document matching the specified criteria.
query = {
'ResourceID': 'spase://SMWG/Document/HPDE/Policy/HP_DataPolicy_v1.2',
'DOI': '10.21978/P83P78',
'Description': '"Program Data Management Plan"'
}
result = hdp.get_spase_document(query)
if result['HttpStatus'] == 200:
print('Result Documents:')
for document in result['Result'].findall('.//Document',
namespaces=NS):
print('ResourceName: ', document.find('.//ResourceName',
namespaces=NS).text)
description = document.find('.//Description',
namespaces=NS)
print(f'Description: {description.text}')
url = document.find('.//AccessInformation/AccessURL/URL',
namespaces=NS)
print(f'URL: {url.text}')
Result Documents: ResourceName: NASA Heliophysics Science Data Management Policy v. 1.2 Description: NASA Heliophysics document that provides guidance on all aspects of HP data, including what needs to go into a Program Data Management Plan, the structure and use of archives, required formats, and a description of the data registry, all in the context of the data lifecycle. URL: https://cdaweb.gsfc.nasa.gov/pub/documents/HPDE/Heliophysics_Data_Policy_v1.2_2016Oct04_signed.pdf
The following code demonstrates how to get the SPASE Software matching the specified criteria.
query = {
'ResourceID': 'spase://CCMC/Software/Kamodo',
'CodeLanguage': 'Python',
'Description': '"space weather models and data"'
}
result = hdp.get_spase_software(query)
if result['HttpStatus'] == 200:
print('Result Software:')
for software in result['Result'].findall('.//Software', namespaces=NS):
print(software.findall('.//ResourceID', namespaces=NS)[0].text)
print(f' {software.findall(".//ResourceName", namespaces=NS)[0].text}')
Result Software:
spase://CCMC/Software/Kamodo
CCMC Kamodo Software
The following code demonstrates how to get the SPASE document with the specified ResourceID.
resource_ids = [
'spase://NASA/Collection/IRIS_AIA',
'spase://SMWG/Service/CCMC/Models'
]
result = hdp.get_spase(resource_ids)
if result['HttpStatus'] == 200:
print('get_spase Result ResourceIDs:')
for spase in result['Result'].findall('.//Spase',
namespaces=NS):
print('ResourceID: ', spase.find('.//ResourceID',
namespaces=NS).text)
print('ResourceName: ', spase.find('.//ResourceName',
namespaces=NS).text)
if 'Last-Modified' in result:
last_modified = result['Last-Modified']
print(f'last_modified = {last_modified.isoformat()}')
get_spase Result ResourceIDs: ResourceID: spase://SMWG/Service/CCMC/Models ResourceName: CCMC Modelweb page ResourceID: spase://SMWG/Service/CCMC/Models ResourceName: CCMC Modelweb page last_modified = 2024-01-11T11:39:22+00:00
The following code demonstrates how to get a SPASE document only if it has been modified since the last time it was gotten.
if last_modified is not None:
result = hdp.get_spase(resource_ids, if_modified_since=last_modified)
if result['HttpStatus'] == 304:
print(f'get_spase if_modified_since {last_modified.isoformat()} return Not Modified')
else:
print(f'get_spase if_modified_since returned an unexpected result {result["HttpStatus"]}')
get_spase if_modified_since 2024-01-11T11:39:22+00:00 return Not Modified
The following code demonstrates how to view an error message from the server in response to a bad request from the client.
query = {
'ResourceID': ['spase://NASA/NumericalData/ACE/CRIS/L2/P1D'],
'Description': '~bad'
}
result = hdp.get_spase_data(types, query, time_range)
if result['HttpStatus'] != 200:
print(f'hdp.get_spase() failed with status = {result["HttpStatus"]}')
if 'ErrorMessage' in result:
print(f'ErrorMessage = {result["ErrorMessage"]}')
print(f'ErrorDescription = {result["ErrorDescription"]}')
#else:
#print(f'HttpText = {result["ErrorText"]}')
print('Display of actual HTML page returned:')
display(HTML(result['ErrorText']))
hdp.get_spase() failed with status = 400 ErrorMessage = Bad Request ErrorDescription = Invalid Description syntax. Display of actual HTML page returned:
Message: Bad Request
Description: Invalid Description syntax.
See Apache Lucene for a description of the search syntax.
Contact 📧 NASA-SPDF-Support if additional help is required.
View the hdpws API documentation for a description of additional features.
