Neurosymbolic Programming Tutorial (POPL23)#

Open In Colab

This is the material used in the Neurosymbolic Summer School 2022 and for the POPL 2022 tutorial on Neurosymbolic Programming. The goal of the notebooks is to provide a hands-on component (30 min each) to complement the tutorials. We provide code as an initial walk-through of baseline methods grounded in behavior analysis applications, with optional exercises for open-ended exploration. The dataset consists of thousands of frames of trajectory data annotated with behavior labels by domain experts. The notebooks demonstrate neurosymbolic programming for behavior quantification, where the task is to learn the relationship between pose and behavior.

Setup#

This tutorial is divided into three Jupyter notebooks that we will run on Google Colab. If you do not wish to save progress, no setup is required. Head over to Notebook 1 to get started!

(Optional) Setup on Google Drive#

If you wish to save your work, we recommend manually copying the provided code to your google drive as shown here: setup.gif

Specifically:

  1. Visit the code repository.

  2. Click on Code (Green Button) > Download ZIP. A file Neurosymbolic_Tutorial-popl23.zip should be downloaded.

  3. Extract the zip file and rename the folder to from Neurosymbolic_Tutorial-popl23 to Neurosymbolic_Tutorial.

  4. Upload the folder to the root folder in your Google Drive.

  5. Navigate to Neurosymbolic_Tutorial/neurosymbolic_notebook{1/2/3}.ipynb.

  6. Set WITHIN_GDRIVE to True in the notebook.

Asking for Help#

Feel free to reach out to Atharva Sehgal. You can reach out to me in three ways:

  1. In Person at POPL23: Flag me down any time during the conference!

  2. Via email: Reach out at atharvas@utexas.edu.

  3. Via Github Issues: Open a GitHub Issue here briefly describing the issue and we can try to debug problems together.

Notebook 1#

Open In Colab

The goal of this notebook is to provide a walk-through of the data with example code for training neural networks and programs.

  • Data Visualization

    • Plot trajectory samples

  • Neural Network

    • Train a 1D Conv Net

  • Program

    • Train program given structure

  • Visualize Model Weights

  • Open-Ended Exploration

Notebook 2#

Open In Colab

This notebook walks through top-down type-guided enumeration, one approach for learning neurosymbolic programs.

  • Running Enumeration

  • Visualize Runtime vs. Classification Performance

  • Implement Temporal Filter

  • Open-Ended Exploration

Notebook 3#

Open In Colab

This notebook walks through informed search via admissible neural heuristics (NEAR), another approach for learning neurosymbolic programs.

  • Running NEAR

  • Visualize Runtime vs. Classification Performance

  • Choose your path:

    • Open-ended Exploration

    • Modifying Architecture of Neural Heuristic

    • IDDFS Search

    • Test on Other Behaviors