CHTC - HTMap Workshop

September 16, 2019

2:00pm - 5:00pm

Instructors: Josh Karpel

Helpers: Christina Koch, Lauren Michael

If you are a researcher using CHTC's high throughput computing (HTC) system to submit and run jobs, we welcome you to attend this workshop on the HTMap package. HTMap is a tool that can help you: You do not have to use Python to attend! If you have some programming experience and use the HTC system, you are welcome to come.

Register Here

General Information

Where: Orchard View Room, Discovery Building, 330 N. Orchard St.. Get directions with OpenStreetMap or Google Maps.

When: September 16, 2019. Add to your Google Calendar.

Requirements: If you want to follow along, see the setup instructions (listed below).

Accessibility: We are committed to making this workshop accessible to everybody. The workshop organizers have checked that:

Contact: Please email ckoch5@wisc.edu for more information.

Schedule

2:00-2:40 Intro and Demonstration
2:40-3:10Questions and Break
3:10-4:00Extended Example
4:00-5:00 Optional Question Time with Josh

Notes

During the workshop, we will put notes and resources into this Google Doc: Google Doc for Notes


Setup

If you would like to follow along with the instruction via your own CHTC account, follow the instructions to set up the necessary tools.

Installing Python and HTMap

This guide will walk you through installing a personal (i.e., not “system”) Python on a CHTC submit machine. Once you have a personal Python, you can install any packages you’d like, including HTMap.

In particular, we will install a self-contained Python distribution called Anaconda. The particular flavor we will install is called Miniconda: it is a bare-bones Python, with just a few packages pre-installed. It comes with the conda package manager in addition to the standard pip.

WARNING: Following these steps will install Python in such a way that it prevents your ability to access the system Python. If you rely on the system Python in your own work, contact us so we can work out a solution.

Log into your submit machine

First, you should log into your submit machine by whatever your normal login mechanism is (probably something like SSH or Putty).

Go to your home directory.

Download the Miniconda installer

We can download the Miniconda installer from the command line. Run

$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

(remember, you don’t type the $, it’s just indicating the command line prompt).

You should see some output from wget. Once wget finishes downloading the file, run ls -l to check that you have a file named Miniconda3-latest-Linux-x86_64.sh in your home directory.

Run the Miniconda installer

Start the installer by running

$ bash Miniconda3-latest-Linux-x86_64.sh

Follow the instructions.

Check that the Miniconda installation worked

The installer has put the files in the right place, but we can’t actually use them yet. Do one of the following:

  1. Run source ~/.bashrc
  2. Log off of the submit machine, then log back in. Once you have done one of the above, there are a few tests we can do to make sure the installation worked correctly.

First, run conda --help. You should see a message like

(base) [karpel@jupyter0000 ~]$ conda --help
usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

conda is the Anaconda package manager. You can use it to install packages distributed by Anaconda, which include various Python packages like numpy or matplotlib that you could also install via pip (Python’s own built-in package manager), but also include a wide variety of other tools.

Next, run these two commands:

$ which python
$ which pip

You should get output like this:

(base) [karpel@jupyter0000 ~]$ which python
~/miniconda3/bin/python
(base) [karpel@jupyter0000 ~]$ which pip
~/miniconda3/bin/pip

The paths should point into the installation directory you gave to the installer above (~/miniconda3 being the default).

Install IPython

The IPython REPL (read-eval-print-loop) is a way to run Python code interactively, without writing actual script files. We’ll use it throughout the workshop. To install it, run

$ conda install ipython

It will ask you for confirmation to proceed after figuring out which packages to install.

Check that IPython works

To check that IPython installed correctly, begin an IPython session by running ipython. You are now in a Python “REPL” (read-eval-print-loop) and can run Python commands. For now, just run something like

>>> 1 + 1
2

If you see the result (2) print out, you’re ready to move on.

If instead you got something like

The program 'ipython' is currently not installed. You can install it by typing:
sudo apt install ipython

do not follow the instructions in that error message; instead, make sure you followed the instructions in this document correctly.

Install HTMap

HTMap can be installed via pip. Run

$ pip install htmap

You should see output from pip as it works; it will not ask you for confirmation.

Check that HTMap works

To check that HTMap installed correctly, begin an IPython session by running ipython from the command line. Run the following command in the REPL:

>>> import htmap; print(htmap.version())

You should see output like HTMap version 0.4.3 (numbers may not match). If you did, HTMap is successfully installed, and you’re ready for the workshop!