Earth Observation Group (EOG) had been providing its products free to the public with direct HTTP link, some of them are secured by Basic Authentication. With increasing popularity of these datasets, EOG will begin securing its data distribution with modern standards. Users will need to register a free account with verified e-mail address to access EOG's resources.
Users can use this link and follow the register link to register their account and test the connection. Once completed, test the login with this link. You shall see the browser displaying welcome message.
The new secured access is easy with browser. You can try this link to experience the flow, and create your account if you do not have one yet.
For those need to perform programmed downloads, the new system uses OAuth2.0 protocol. Users need to use Bearer token in the request header to gain access to the file. Below are some examples to implement programmed access.
To retrieve token from authentication server, user need to supply the following key-param pairs in the request body.client_id=eogdata_oidc
client_secret=2677ad81-521b-4869-8480-6d05b9e57d48
username=<your_username>
password=<your_password>
Use the following command to retrieve the token, parse it as JSON format and save to $response variable.response=`curl -L -d 'client_id=eogdata_oidc' -d 'client_secret='2677ad81-521b-4869-8480-6d05b9e57d48' -d 'username=<username>' -d 'password=<password>' -d 'grant_type=password' 'https://eogauth.mines.edu/auth/realms/master/protocol/openid-connect/token' | python -m json.tool`
Then the token string need to be parsed to extract the JWT (JSON Web Token) access token and save to $access_token variable. access_token=`echo $response|jq '.access_token'|sed 's/^.//;s/.$//'`
Finally, the token can be as a Bearer Token to access protected resources. The access token is good for 24 hours. It is a good idea to renew the token before it expires. curl -L https://eogdata.mines.edu/eog/EOG_sensitive_contents -H "Authorization: Bearer $access_token"
import requests
import json
import os
# Retrieve access token
params = {
'client_id': 'eogdata_oidc',
�� 'client_secret': '2677ad81-521b-4869-8480-6d05b9e57d48',
'username': <username>,
'password': <password>,
'grant_type': 'password'
}
token_url = 'https://eogauth.mines.edu/auth/realms/master/protocol/openid-connect/token'
response = requests.post(token_url, data = params)
access_token_dict = json.loads(response.text)
access_token = access_token_dict.get('access_token')
# Submit request with token bearer
## Change data_url variable to the file you want to download
data_url = 'https://eogdata.mines.edu/eog/EOG_sensitive_contents'
auth = 'Bearer ' + access_token
headers = {'Authorization' : auth}
response = requests.get(data_url, headers = headers)
# Write response to output file
## You can either define the output file name directly
# output_file = 'EOG_sensitive_contents.txt'
## Or get the filename from the data_url variable
output_file = os.path.basename(data_url)
with open(output_file,'wb') as f:
f.write(response.content)
library(httr)
library(jsonlite)
library(utils)
# Retrieve access token
params <- list(
client_id = 'eogdata_oidc',
client_secret = '2677ad81-521b-4869-8480-6d05b9e57d48',
username = <username>,
password = <password>,
grant_type = 'password'
)
token_url <- 'https://eogauth.mines.edu/auth/realms/master/protocol/openid-connect/token'
response <- POST(token_url, body = params, encode = "form")
access_token_list <- fromJSON(content(response,as="text",encoding="UTF-8"))
access_token <- access_token_list$access_token
# Submit request with token bearer and write to output file
## Change data_url variable to the file you want to download
data_url <- 'https://eogdata.mines.edu/eog/EOG_sensitive_contents'
auth <- paste('Bearer', access_token)
## You can either define the output file name directly
# output_file <- 'EOG_sensitive_contents.txt'
## Or get the filename from the data_url variable
output_file <- basename(data_url)
download.file(data_url,output_file,mode = "wb", headers = list(Authorization = auth))
Users can use this link to