Workflow for data replication into the CDP¶

the steps should be followed by persons responsible for data ingest into the DKRZ CMIP data pool (CDP) via ESGF replication

configuration changes as described require membership in the DKRZ-ESGF group (group 933), please contact Stephan for membership requests

Setup and configure replication software¶

• load synda replication software module
• documentation of synda: https://github.com/Prodiguer/synda
  

  module load synda
  

• set Path to synda home
  

  export ST_HOME=/sw/rhel6-x64/dm/sdt
  

• set your credentials
  (can be omitted in this case default credentials are used)
  

  cd $ST_HOME/conf
  vi credentials.comf
  

• default configuration settings
  • synda database and target directory:
    these are set in $ST_HOME/conf/sdt.conf:
    

    db_path = /sw/rhel6-x64/dm/sdt/db
    data_path = /work/kd0956/Test
    
    change to:
    data_path = /work/kd0956/"myproject"/data  
    

setup and configure your replication job¶

• edit a replication description file in the specific selection folder
  

  cd ST_HOME
  cd selection/CMIP5 
  vi name_of_specific_replication_job.txt
  

check and submit the replication job¶

• commit the selection/CMIP5 repo
  

  cd $ST_HOME/selection/CMIP5
  git add .
  git commit -m "my comment" 
  

• renew your credentials and submit replication job
  

  synda daemon start (only if synda daemon is not running)
  synda certificate renew
  synda install $ST_HOME/selection/CMIP5/name_of_specific_replication_job.txt
  

• monitor running replication jobs
  

  synda history
  ...