2021#

Dec 1st 2021#

Attending#

  • Aakash / Australian National University / aakashgc

  • Chris H / 2i2c / choldgraf

  • Rowan / Curvenote / rowanc1

  • Damián / 2i2c / damianavila

  • Chris S / EPFL / chrisjsewell

Apologies:

  • Matt McKay / Australian National University / mmcky (Some updates below @Aakash can speak to them if needed)

Team updates#

  • [sphinx-exercise] has been re-factored to make the extension more maintainable.

    • executablebooks/sphinx-exercise#37

    • no major user facing changes (except some style updates to output)

    • focus: use sphinx internals and AST as much as possible

    • comments: found numref integration pretty hard to get right as numref is implemented (in part) in the translator phase

    • reviews: Welcome any and all comments, also would love to chat with anyone with knowledge on Sphinx references (ref and numref)

    • preview: https://61a58d957549766565b7eb2a–peaceful-feynman-936ca4.netlify.app/intro.html

    • todo: write developer notes and lessons learnt for a general “admonition factory” with support for:

      • custom styling

      • custom options

      • ref and numref integration

      • autonumbering of nodes (i.e. numref)

  • Rowan

    • Javascript PR merges in

    • Low hanging fruit is done on the JS side

  • Chris S

    • Working on VSCode, then JupyterLab integration with the MyST JS parser

  • Steve

    • Thebe - underlying infrastructure and moving away from JQuery

  • Damian

    • Working on 2 major PRs from the PyData Sphinx Theme

  • Aakash

    • Hasn’t had time to work on EBP stuff much

    • Interested in working on JTex for single-page PDFs

Agenda items#

  • Discuss using jtex to build single-page PDFs of pages

    • A quick low-hanging fruit would be to improve the CSS for printing pages in the book theme.

    • However this won’t satisfy most needs for publishing articles etc, so that would be a good usecase for jtex to tackle after dealing with low-hanging fruit.

    • executablebooks/sphinx-book-theme#435

October 3rd 2021#

Attending#

  • Rowan C / Curvenote / rowanc1

  • Steve Purves / Curvenote

  • Chris H / 2i2c / choldgraf

  • Anton Akhmerov / TU Delft / akhmerov

  • Chris S / EPFL / chrisjsewell

  • John S. / ANU

  • Matt M. / ANU

Team updates#

  • Chris H

    • Community strategy for the EOL on our Sloan grant: executablebooks/meta#493

    • Focusing most of my time trying to document things + un-block people

    • Have played around with Pradyun’s sphinx-basic-ng theme and really like it

  • Steve P

    • Progress on thebe:

      • Major styles bug fixed

      • First cut of status and activate button out

      • Changed the docs & examples to use latest build - Thanks to recent RTD changes

      • Started a Jupyterlite example - have JupyterLiteServer running in browser

    • Next Steps:

      • Test & Release

        • we need to get an update out, as some projects are pinning earlier releases because of that CSS bug. (will coordinate with Chris H.)

      • Refactoring

        • separate jquery layer from low level api (opens up more options for integration)

        • document jquery and low level api

        • follow through on rename

      • Include Jupyterlite kernel option (and generalise)

      • Tackle enhancements from Chris S. in thebe / sphinx-thebe

  • Matt M

    • Sphinx V4 Support in executablebooks/jupyter-book#1448

    • Refactor of sphinx-exercise (include LaTeX support for nodes) in final testing

      • Would it be useful to have a generalized function / tool to quickly create semantic admonitions that are well-structured? Common Infrastructure? Might be a good way to document admonition -> theme style relationships.

      • General agreement that this would be useful :-)

    • Added LaTeX support to sphinx-proof

    • Semantic Admonitions!

  • Chris S

    • +1 for sphinx-basic-ng and merging focus of themes

    • Participating in https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525

      • Made PoC PR to nbclient and nbformat

      • Participated in Jupyter Community Meeting and spoke after with Angus Goose and Tony Fast and maybe set up further talks

      • But will it go stale 😬

    • Added skip-execution cell tag functionality in nbclient: jupyter/nbclient#151

    • Created myst-nb documentation for sqlalchemy tutorial and fed back to thebe

    • Working with cpitclaudel to add “proper” docutils support for myst-parser

    • Working with tanelli to update markdown-it-py to latest version of markdown-it (and finally fix the more complex url parsing)

    • Will also eventually get back to improving jupyter-cache (executablebooks/jupyter-cache#74) and long awaited myst-nb “refactor”

  • Rowan

    • Have been working on some other open source libraries for templating and exporting to word/latex/pdf from myst.

    • Need to get back to myst in js work, next week!

      • As implementation starts to happen on the TS/JS/Curvenote side, then it will open opportunities to have discussions around the “core vocabulary” of MyST - what admonitions should be supported everywhere

Agenda items#

Activity board#

Community proposal doc#

  • Issue for comments/discussion: executablebooks/meta#493

  • Proposal text: https://hackmd.io/@choldgraf/HkgAIHtQY

  • What are some major next steps?

    • Investigate the major steps we’d need to take to integrate with the Jupyter community

      • Sticking points?

        • We’ll need to demonstrate that we have a good maintainability story

        • Might have questions around the different parts of EBP

      • Start to get feedback in the Jupyter ecosystem

        • Threads in the discourse + the governance repo

        • Talk to stakeholders in that community and see if there are any things we should think about it

    • Crystallize the community strategy doc into some concrete next steps

      • Decide on an order of operations for things to tackle

September 1st 2021#

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • Name / Affiliation / Handle

  • Steve Purves / Curvenote / stevejpurves

  • Rowan Cockett / Curvenote / rowanc1

  • Matt McKay / ANU / mmcky

  • Chris Holdgraf / 2i2c / choldgraf

  • Damián Avila / 2i2c / damianavila

  • Chris Sewell / EPFL / chrisjsewell

Reports, updates, and celebrations#

This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • THEBE:

    • Some Thebe PRs in recently, and refactoring work started, other PRs ongoing

    • Opened(opening) tickets on activity board for dicussion

    • Refactoring, improving test coverage and docs

    • Extended kernel support for Juptyerlite/pyodide, local proxy and better kernel selection

  • Some blog post about MyST in my personal blog (Damián)

    • Part 1

    • Part 2

    • Eventually, a part 3 with some coding experiment/stuff

  • Sphinx 4 Upgrade (Matt)

  • ReadTheDocs integration (Chris S)

    • That is now working

    • PR is open - just waiting on documenting it up

    • Action: Needs Doc Update

Agenda items#

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • Steve: Thebe directions, and future focus

    • Setup a Project board on the thebe repo to better plan out work there, in more detail that the Activity Board maybe?

    • A lot of issues in the repository

      • Was a JupyterCon Sprint + a board from that

    • Would like to do a bit of planning about what issues etc to tackle next

    • There is definitely the opportunity now to move a bit more quickly and make changes before opening up for contributions

    • Priority definely to get the CSS issue fixed so jupyter-book can move to the latest

  • Rowan: Thoughts on how to move JS-Myst forward (dev branch?)

    • Few-no current dependencies on the work, and could possibly merge/deploy faster than some other repos.

    • Might just be an off month?!

    • Is there a way that we can ensure a consistent momentum of PRs so that we don’t constantly block on people’s reviews etc?

    • Right now we are at an important early design moment on the JS side. Longer term it should be faster as the PRs will be less complex.

    • In the meantime, Chris S will prioritize his EBP stuff on making sure these PRs get reviewed.

  • Chris H: in-line markdown execution?

    • Long conversation about this here: https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/37

    • The difficulty is in the specification

    • mmcky: Related to adding execution to exercise nodes (which we are prototyping) and working out how to get execution embedded in directives. Current approach is through gated directives (which wouldn’t work obviously for roles)

    • An easy first step would be to do “read only inline markdown”

    • This is a really popular feature of RMarkdown / Quarto Markdown

    • Can you start with the restriction “we should be able to still run every code cell”

    • From a user’s perspective, you’d just want them to be able to put in variables with some kind of syntax e.g. {{ }}

      • You’d need to go through the markdown cells and look for the {{ }} syntax and know what kinds of variables that you need.

      • You’d then do the execution, and at the end of the time that

05th August 2021#

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • Matthew McKay / ANU / mmcky

  • Steve Purves / Curvenote / stevejpurves

  • Chris Sewell

  • Rowan Cockett / Curvenote / rowanc1

  • Damián Avila / 2i2c / damianavila

Reports, updates, and celebrations#

This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.

  • Chris Sewell

    • Sphinx Designs - better designed panels, will come into jupyter book at some point

    • Jupyter Cache (for executing notebooks –> myst-nb –> JupyterBook) - rewritten

      • Decides if code cells are changed and caches outputs. Then can grab the outputs from a local database. There is a CLI!

      • There is also multi-processing on the notebooks! 💥

      • There is a new PR draft

    • Javascript!

      • Added a new repo docutils

      • There are same directives and roles from Python

        • Same arguments/content

      • This is now over to the vscode extension

    • MyST as a “standard”

      • What roles, directives, blocks to we need that are copied over in another language

      • We should get the critical ones in a list somewhere!

        • Forming specs in a collaborative way (Identify Good Tools for this?)

  • Matt McKay: Taken over maintaining sphinx-proof and sphinx-exercise (looking into enabling execution through gated directives as a proof of concept)

    • “enabling execution in directives” (executablebooks/sphinx-exercise#24)

      • Adding a directive that is a question, and then how do you add code?

        • Directive at the top level and then code and exercise.

          • Violates the core design of the Jupyter Notebook – nested code cells

          • Got rid of “nested” concepts

        • So how do you have “Gated” directive (open/close) around a few code cells

          • This is more compliant with other tools like Jupytext

        • Not sure if there is a Gated directive, or prior art in the Sphinx world?

      • Other options:

        • Metadata? Adding tags, etc. (Similar to nbgrader notebooks)

        • Glue? (currently restricted to python, cather than cellId or something)

          • Similar ideas playing with at Curvenote! (copy/paste)

      • Consideration is mostly from an authorship perspective

  • Steve Purves: Thebe

    • got started and made some improvements to the build and docs – yarn, CI issues, local kernel convenience commands for development

    • PR for a small feature - progress indicator

    • Collaborating a bit with LibreText folks, would like to more and understand how they are using Thebe better

    • LibreText made a big change to utilise the @jupyterlab/manager in place of a shim+custom code – so waiting on that PR coming in before pushing on

    • Next up:

      • Connecting to many different kernals, and having some basic management on that!

  • Rowan: MyST in JS

    • Working on myst in JS. Migrating work over to docutils and cleaning up along the way

    • Still basics of various libraries

    • 10 or so PRs up at the moment!

      • Basics:

        • sub/sup/abbr

        • internationalization? [each role/directive has a different name]

          • Can generate various dicts to export

      • Admonitions

      • Math

        • amsmath, dollar, math directive

      • Figures

      • Referencing/numbering PR

        • numref/eq/ref

    • Working to remove all of the markdown-it-myst functionality –> docutils w/ review from ChrisS

    • How are we working with styles?

      • There is some copying back and forth of the .css

  • Damián

    • Learning Myst internals working in the Nikola + Myst story

    • Probably writing a series of blog posts to push discussion forward about things I have found and engage the community

    • MyST as a “standard” > What roles, directives, blocks do we need copied over Python (independent from Docutils/Sphinx)? Do we need a docultils/sphinx free Py representation of the “standard”?

Agenda items#

Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.

  • NAME: ITEM (EXPECTED TIME TO DISCUSS)

  • Discuss team activity board proposal (executablebooks/meta#441)

    • The goal of this board is to coordinate and plan our major work items, to make it clear who is working on what, and to help signal boost items for review/discussion.

    • Is this board structure worth trying as-is or are there major changes to make to it?

    • If we want to try this process, perhaps we should use this meeting to add some initial items to the board?

Discussion:

  1. This was discussed with the team and we agreed to use it this month to see how it works.

  2. Any suggestions to simplify or improve the Activity Board was encouraged.

  3. We focused on keeping the activity board pretty high level for cross-cutting communication and to help coordinate.

  4. We also discussed keeping an eye on the Request for Review board to get visibility on PR’s that need review and get more distributed reviews across the team.

  5. We should add the Activity Board as a standing discussion point for the monthly meeting

  • Steve: Next Steps for Thebe

    • Refactoring the one big file into more manageable chunks

    • More tests - current test coverage is poor, PRs go green but not much really covered which is maybe worse than no tests at all

    • Kernel Status & Management - as part of refactor want to make it easier to get status of the kernel and lay ground work for the issues asking for: Kernel Status, Multiple Kernels and Jupyterlite connectivity; 412, 349, 271

Discussion:

  1. This was discussed in the Reports section.

July 1st, 2021#

If you are joining the team video meeting, sign in below so we know who was here. Roll call:

  • Matthew McKay / Australian National University (ANU) / mmcky

  • Chris Sewell / EPFL / chrisjsewell

  • Steve Purves / Curvenote / stevejpurves

  • John Stachurski / ANU / jstac

  • Chris Holdgraf / 2i2c / choldgraf

  • Damián Avila / 2i2c / damianavila

  • Aakash Gude / ANU / AakashGfude

Reports, updates, and celebrations#

This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.

QuantEcon / ANU#

  • QuantEcon: 4/5 Projects running Jupyter Book (remaining: julia)

  • ANU Team: Working on Support for Individual Page PDF (via LaTeX) generation (Lead: @mmcky)

    • Chris H notes that we should include updates about QuantEcon lectures conversion etc

    • @mmcky to add links to executable books gallery to QuantEcon jupyter-book powered projects

Chris S.#

Recent work

  • Released sphinx-external-toc and integrated in Jupyter-Book

    • Removes a lot of code from jupyter-book 😄, moving towards JB just being a thin wrapper for sphinx-extensions (last part is basically the _config.yml)

    • Bit of a balancing act between; ease of use/understandanding, having a clear schema, having all the functionality of sphinx toctrees

    • Integration in JB also includes addition of “pluggable” CLI

  • Released markdown-it-py v1.0, i.e. fully stable

  • Released myst-parser v0.15.0, with sphinx v4 support

    • Should be moving all repositories to sphinx v4

  • Released rst-to-myst v0.3

    • Facile conversion of RST to MyST Markdown for a whole project

    • Utilises mdformat + mdformat-myst to create the Markdown from markdown-it tokens

    • Still working on mdformat-myst, but this essentially allows round trips Markdown text -> markdown-it tokens -> Markdown text. Could potentially be used for e.g. modifying links in downloadable notebooks

  • Created markdown-it-plugin-template; template repository with best practices for TypeScript library

Intended work

  • Update MyST VS Code extension to use new markdown-it plugins

  • Create sphinx-design: An iteration on sphinx-panels, but with the benefit of hindsight, and to more closely follow aspects of Materials Design and Material-UI

    • class names that do not clash with sphinx ones (i.e. container)

    • easier to set principle colors and width defaults

  • Either in myst-parser or sphinx-design have nice way to generate stylised post-header banners, with e.g. author, date, read time (see executablebooks/MyST-Parser#372)

  • Improvements to myst-nb (+ jupyter-cache)

    • Been gradually working on it, but lot of interconnected parts 😬

    • Revise the AST and transforms (post-transform -> transform)

    • Drop nbconvert, jupyter-sphinx dependencies

    • Better support for Markdown code output

    • Better support for “multi-level” configuration (cell level > notebook level > conf.py level)

    • Improve notebook execution and (maybe) drop auto mode in favour of cache

      • Support for dynamic kernel names

    • How to handle glue; ideally in a kernel agnostic manner, possily with the new substitution syntax

    • Plugin CLI in jupyter-book for notebook execution

Steve#

  • Would like to:

    • meet the team

    • hear about thoughts on Thebelab as I’m prepping to do some work there

Berkeley team#

  • :wave: to Damian, onboarding etc

  • Damian is playing around with the MyST parser and learning how it is structured, experimenting with a Nikola implementation of the parser.

  • Chris is going to focus more of his time on team documentation, tool documentation for both users and developers, etc.

Agenda items#

Didn’t get to the following items

  • Bug issues in Jupyter Book: any important issues to address from: executablebooks/jupyter-book#issues

  • Jupyter-book in RTD: how can we make it happen!?

    • Chris H can give a short update from a conversation he had with Eric and Juan at RTD a few days ago.

  • Theming: I am still keen on creating a shared infrastructure (executablebooks/meta#279) and not necessarily being directly dependent on pydata-sphinx-theme

    • Although @choldgraf mentioned about possible funding from NumFocus for pydata-sphinx-theme

  • MyST as specification: what does that mean, how to “de-couple” from sphinx?

    • What roles/directives/extensions are “core”?

    • How to support additional roles/directives/extensions

    • How does this tie in with “JavaScript” work (i.e. editor tools in VS Code, JupyterLab, etc)?

    • Some notes in executablebooks/markdown-it-myst

John S:

  • Hiring casual RAs?

Chris H:

  • Communicating the work we’re doing - blog posts, tweets, etc? @executablebooks handle?

  • Coordinating around deliverables, workstreams, requests for reviews, etc?