Earth Observation Group (EOG) had been providing its products free to the public with direct HTTP link, some of them are secured by Basic Authentication. With increasing popularity of these datasets, EOG will begin securing its data distribution with modern standards. Users will need to register a free account with verified e-mail address to access EOG's resources.
Users can use this link to register an account and test the connection. Once completed, users should see the the following welcome message:
For paid subscribers who need to perform programmatic downloads, the new system uses OAuth2.0 protocol. Users will need to use a Bearer token in the request header to gain access to the file. Below are some examples to implement programmatic access.
To retrieve token from authentication server, user need to supply the following key-param pairs in the request body.client_id=<your client id>
client_secret=<your_client_secret>
username=<your_username>
password=<your_password>
grant_type='password'
Use the following command to retrieve the token, parse it as JSON format and save to $response variable.response=`curl -L -d 'client_id=<your client id>' -d 'client_secret=<your_client_secret>' -d 'username=<username>' -d 'password=<password>' -d 'grant_type=password' 'https://eogauth-new.mines.edu/realms/eog/protocol/openid-connect/token' | python -m json.tool`
Then the token string need to be parsed to extract the JWT (JSON Web Token) access token and save to $access_token variable. access_token=`echo $response|jq '.access_token'|sed 's/^.//;s/.$//'`
Finally, the token can be as a Bearer Token to access protected resources. The access token is good for 5 minutes. It is a good idea to renew the token before it expires. curl -L https://eogdata.mines.edu/eog/EOG_sensitive_contents -H "Authorization: Bearer $access_token"
import requests
import json
import os
# Retrieve access token
params = {
'client_id': '<your client id>',
'client_secret': '<your_client_secret>',
'username': <username>,
'password': <password>,
'grant_type': 'password'
}
token_url = 'https://eogauth-new.mines.edu/realms/eog/protocol/openid-connect/token'
response = requests.post(token_url, data = params)
access_token_dict = json.loads(response.text)
access_token = access_token_dict.get('access_token')
# Submit request with token bearer
## Change data_url variable to the file you want to download
data_url = 'https://eogdata.mines.edu/eog/EOG_sensitive_contents'
auth = 'Bearer ' + access_token
headers = {'Authorization' : auth}
response = requests.get(data_url, headers = headers)
# Write response to output file
## You can either define the output file name directly
# output_file = 'EOG_sensitive_contents.txt'
## Or get the filename from the data_url variable
output_file = os.path.basename(data_url)
with open(output_file,'wb') as f:
f.write(response.content)
library(httr)
library(jsonlite)
library(utils)
# Retrieve access token
params <- list(
client_id = '<your client id>',
client_secret = '<your_client_secret>',
username = <username>,
password = <password>,
grant_type = 'password'
)
token_url <- 'https://eogauth-new.mines.edu/realms/eog/protocol/openid-connect/token'
response <- POST(token_url, body = params, encode = "form")
access_token_list <- fromJSON(content(response,as="text",encoding="UTF-8"))
access_token <- access_token_list$access_token
# Submit request with token bearer and write to output file
## Change data_url variable to the file you want to download
data_url <- 'https://eogdata.mines.edu/eog/EOG_sensitive_contents'
auth <- paste('Bearer', access_token)
## You can either define the output file name directly
# output_file <- 'EOG_sensitive_contents.txt'
## Or get the filename from the data_url variable
output_file <- basename(data_url)
download.file(data_url,output_file,mode = "wb", headers = list(Authorization = auth))
Users can use this link to