V-1. Download example dataset and notebooks

  • Login to FLUX and set up the same path we used yesterday when exploring genomic data formats
    export PATH=/home/hmkang/bioboot/bin:${PATH} 
    export PATH=/scratch/biobootcamp_fluxod/remills/anaconda/bin:${PATH}
  • Navigate to your bootcamp scratch directory (cd /scratch/biobootcamp_fluxod/UNIQUENAME). If a directory with your uniquename does not exist, make it using

    mkdir /scratch/biobootcamp_fluxod/UNIQUENAME
    cd /scratch/biobootcamp_fluxod/UNIQUENAME
  • Once in your directory, create a subdirectory for today’s segment

    mkdir day4_cluster
    cd day4_cluster
  • download the data for today to FLUX
    • Note that you can view a render of this notebook directly on GitHub as well!
    wget https://github.com/bioboot/web-2015/raw/gh-pages/class-material/read_counts_by_region.tar.gz
    tar zxvf read_counts_by_region.tar.gz
    cd read_counts_by_region
  • For this exercise, we will need to run ipython notebook on flux. As with yesterday, start a notebook server with the following commands

    • Start a notebook server
    • Notes:
      1. you need to know which host you’re on. The command line prompt will show this (e.g., flux-login3)
      2. you need to know which port your instance of ipython listens to. The first few lines of output from ipython notebook will list this for you.
    remills@flux-login3:/scratch/biobootcamp_fluxod/remills/biobootcamp$ ipython notebook --ip=flux-login3 --no-browser
    [I 13:26:18.660 NotebookApp] Using MathJax from CDN: https://cdn.mathjax.org/mathjax/latest/MathJax.js
    [I 13:26:19.220 NotebookApp] The port 8888 is already in use, trying another random port.
    [I 13:26:19.473 NotebookApp] Serving notebooks from local directory: /scratch/biobootcamp_fluxod/kitzmanj
    [I 13:26:19.473 NotebookApp] 0 active kernels
    [I 13:26:19.473 NotebookApp] The IPython Notebook is running at: http://flux-login3:8889/
    [I 13:26:19.473 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
    • A word of warning: in general, running on the head node of the flux cluster is not a good practice, because the head node controls the job queue for the entire cluster. Instead you would want to enter the queue, login to a compute node, and do your work there.

    • Now, your own ipython server is running on flux. But you will need to open a tunnel to get there. The below command will securely connect port 8889 on the computer flux-login3 to port 9000 on my computer.

     ssh -L localhost:9000:flux-login3:8889 YOURNAME@flux-login.engin.umich.edu
    • Navigate to http://localhost:9000 and make a new notebook
  • Ipython notebook reminders:

    • Green box - marks active cell. You are in EDIT mode.
    • Hit escape to exit mode mode and go to COMMAND mode.
    • In command mode:
      • A : insert new cell above
      • B : insert new cell below
      • Enter : edit the currently selected cell
      • up/down arrow : navigate up or down
    • In either:
      • Shift-Enter : run the current cell
    • Remember, you can run commands out of order in the notebook.