2021#
Dec 1st 2021#
Attending#
Aakash / Australian National University / aakashgc
Chris H / 2i2c / choldgraf
Rowan / Curvenote / rowanc1
Damián / 2i2c / damianavila
Chris S / EPFL / chrisjsewell
Apologies:
Matt McKay / Australian National University / mmcky (Some updates below @Aakash can speak to them if needed)
Team updates#
[sphinx-exercise] has been re-factored to make the extension more maintainable.
no major user facing changes (except some style updates to output)
focus: use sphinx internals and AST as much as possible
comments: found numref integration pretty hard to get right as numref is implemented (in part) in the translator phase
reviews: Welcome any and all comments, also would love to chat with anyone with knowledge on Sphinx references (ref and numref)
preview: https://61a58d957549766565b7eb2a–peaceful-feynman-936ca4.netlify.app/intro.html
todo: write developer notes and lessons learnt for a general “admonition factory” with support for:
custom styling
custom options
ref and numref integration
autonumbering of nodes (i.e. numref)
Rowan
Javascript PR merges in
Low hanging fruit is done on the JS side
Chris S
Working on VSCode, then JupyterLab integration with the MyST JS parser
Steve
Thebe - underlying infrastructure and moving away from JQuery
Damian
Working on 2 major PRs from the PyData Sphinx Theme
Aakash
Hasn’t had time to work on EBP stuff much
Interested in working on JTex for single-page PDFs
Agenda items#
Discuss using
jtex
to build single-page PDFs of pagesA quick low-hanging fruit would be to improve the CSS for printing pages in the book theme.
However this won’t satisfy most needs for publishing articles etc, so that would be a good usecase for
jtex
to tackle after dealing with low-hanging fruit.
October 3rd 2021#
Attending#
Rowan C / Curvenote / rowanc1
Steve Purves / Curvenote
Chris H / 2i2c / choldgraf
Anton Akhmerov / TU Delft / akhmerov
Chris S / EPFL / chrisjsewell
John S. / ANU
Matt M. / ANU
Team updates#
Chris H
Community strategy for the EOL on our Sloan grant: executablebooks/meta#493
Focusing most of my time trying to document things + un-block people
Have played around with Pradyun’s
sphinx-basic-ng
theme and really like it
Steve P
Progress on thebe:
Major styles bug fixed
First cut of status and activate button out
Changed the docs & examples to use latest build - Thanks to recent RTD changes
Started a Jupyterlite example - have JupyterLiteServer running in browser
Next Steps:
Test & Release
we need to get an update out, as some projects are pinning earlier releases because of that CSS bug. (will coordinate with Chris H.)
Refactoring
separate jquery layer from low level api (opens up more options for integration)
document jquery and low level api
follow through on rename
Include Jupyterlite kernel option (and generalise)
Tackle enhancements from Chris S. in thebe / sphinx-thebe
Matt M
Sphinx V4 Support in executablebooks/jupyter-book#1448
Refactor of sphinx-exercise (include LaTeX support for nodes) in final testing
Would it be useful to have a generalized function / tool to quickly create semantic admonitions that are well-structured? Common Infrastructure? Might be a good way to document admonition -> theme style relationships.
General agreement that this would be useful :-)
Added LaTeX support to sphinx-proof
Semantic Admonitions!
Chris S
+1 for
sphinx-basic-ng
and merging focus of themesParticipating in https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525
Made PoC PR to nbclient and nbformat
Participated in Jupyter Community Meeting and spoke after with Angus Goose and Tony Fast and maybe set up further talks
But will it go stale 😬
Added
skip-execution
cell tag functionality in nbclient: jupyter/nbclient#151Created myst-nb documentation for sqlalchemy tutorial and fed back to thebe
Also added merging streams in myst-nb
Might work with them further on their documentation: sqlalchemy/sqlalchemy#7138
Working with
cpitclaudel
to add “proper” docutils support for myst-parserWorking with
tanelli
to update markdown-it-py to latest version of markdown-it (and finally fix the more complex url parsing)Will also eventually get back to improving jupyter-cache (executablebooks/jupyter-cache#74) and long awaited myst-nb “refactor”
Rowan
Have been working on some other open source libraries for templating and exporting to word/latex/pdf from myst.
Need to get back to myst in js work, next week!
As implementation starts to happen on the TS/JS/Curvenote side, then it will open opportunities to have discussions around the “core vocabulary” of MyST - what admonitions should be supported everywhere
Agenda items#
Activity board#
switch to new GitHub project boards? Demo here: executablebooks/projects#9
Community proposal doc#
Issue for comments/discussion: executablebooks/meta#493
Proposal text: https://hackmd.io/@choldgraf/HkgAIHtQY
What are some major next steps?
Investigate the major steps we’d need to take to integrate with the Jupyter community
Sticking points?
We’ll need to demonstrate that we have a good maintainability story
Might have questions around the different parts of EBP
Start to get feedback in the Jupyter ecosystem
Threads in the discourse + the governance repo
Talk to stakeholders in that community and see if there are any things we should think about it
Crystallize the community strategy doc into some concrete next steps
Decide on an order of operations for things to tackle
September 1st 2021#
If you are joining the team video meeting, sign in below so we know who was here. Roll call:
Name / Affiliation / Handle
Steve Purves / Curvenote / stevejpurves
Rowan Cockett / Curvenote / rowanc1
Matt McKay / ANU / mmcky
Chris Holdgraf / 2i2c / choldgraf
Damián Avila / 2i2c / damianavila
Chris Sewell / EPFL / chrisjsewell
Reports, updates, and celebrations#
This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.
THEBE:
Some Thebe PRs in recently, and refactoring work started, other PRs ongoing
Opened(opening) tickets on activity board for dicussion
Refactoring, improving test coverage and docs
Extended kernel support for Juptyerlite/pyodide, local proxy and better kernel selection
Some blog post about MyST in my personal blog (Damián)
Sphinx 4 Upgrade (Matt)
myst-nb PR executablebooks/MyST-NB#356
Action: Cut a new release right after merge
ReadTheDocs integration (Chris S)
That is now working
PR is open - just waiting on documenting it up
Action: Needs Doc Update
Agenda items#
Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.
Steve: Thebe directions, and future focus
Setup a Project board on the thebe repo to better plan out work there, in more detail that the Activity Board maybe?
A lot of issues in the repository
Was a JupyterCon Sprint + a board from that
Would like to do a bit of planning about what issues etc to tackle next
There is definitely the opportunity now to move a bit more quickly and make changes before opening up for contributions
Priority definely to get the CSS issue fixed so jupyter-book can move to the latest
Rowan: Thoughts on how to move JS-Myst forward (dev branch?)
Few-no current dependencies on the work, and could possibly merge/deploy faster than some other repos.
Might just be an off month?!
Is there a way that we can ensure a consistent momentum of PRs so that we don’t constantly block on people’s reviews etc?
Right now we are at an important early design moment on the JS side. Longer term it should be faster as the PRs will be less complex.
In the meantime, Chris S will prioritize his EBP stuff on making sure these PRs get reviewed.
Chris H: in-line markdown execution?
Long conversation about this here: https://discourse.jupyter.org/t/inline-variable-insertion-in-markdown/10525/37
The difficulty is in the specification
mmcky: Related to adding execution to exercise nodes (which we are prototyping) and working out how to get execution embedded in directives. Current approach is through gated directives (which wouldn’t work obviously for roles)
An easy first step would be to do “read only inline markdown”
This is a really popular feature of RMarkdown / Quarto Markdown
Can you start with the restriction “we should be able to still run every code cell”
From a user’s perspective, you’d just want them to be able to put in variables with some kind of syntax e.g.
{{ }}
You’d need to go through the markdown cells and look for the
{{ }}
syntax and know what kinds of variables that you need.You’d then do the execution, and at the end of the time that
05th August 2021#
If you are joining the team video meeting, sign in below so we know who was here. Roll call:
Matthew McKay / ANU / mmcky
Steve Purves / Curvenote / stevejpurves
Chris Sewell
Rowan Cockett / Curvenote / rowanc1
Damián Avila / 2i2c / damianavila
Reports, updates, and celebrations#
This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.
Chris Sewell
Sphinx Designs - better designed panels, will come into jupyter book at some point
Jupyter Cache (for executing notebooks –> myst-nb –> JupyterBook) - rewritten
Decides if code cells are changed and caches outputs. Then can grab the outputs from a local database. There is a CLI!
There is also multi-processing on the notebooks! 💥
There is a new PR draft
Javascript!
Added a new repo docutils
There are same directives and roles from Python
Same arguments/content
This is now over to the vscode extension
MyST as a “standard”
What roles, directives, blocks to we need that are copied over in another language
We should get the critical ones in a list somewhere!
Forming specs in a collaborative way (Identify Good Tools for this?)
Matt McKay: Taken over maintaining sphinx-proof and sphinx-exercise (looking into enabling execution through gated directives as a proof of concept)
“enabling execution in directives” (executablebooks/sphinx-exercise#24)
Adding a directive that is a question, and then how do you add code?
Directive at the top level and then code and exercise.
Violates the core design of the Jupyter Notebook – nested code cells
Got rid of “nested” concepts
So how do you have “Gated” directive (open/close) around a few code cells
This is more compliant with other tools like Jupytext
Not sure if there is a Gated directive, or prior art in the Sphinx world?
Other options:
Metadata? Adding tags, etc. (Similar to nbgrader notebooks)
Glue? (currently restricted to python, cather than cellId or something)
Similar ideas playing with at Curvenote! (copy/paste)
Consideration is mostly from an authorship perspective
Steve Purves: Thebe
got started and made some improvements to the build and docs – yarn, CI issues, local kernel convenience commands for development
PR for a small feature - progress indicator
Collaborating a bit with LibreText folks, would like to more and understand how they are using Thebe better
LibreText made a big change to utilise the @jupyterlab/manager in place of a shim+custom code – so waiting on that PR coming in before pushing on
Next up:
Connecting to many different kernals, and having some basic management on that!
Rowan: MyST in JS
Working on myst in JS. Migrating work over to docutils and cleaning up along the way
Still basics of various libraries
10 or so PRs up at the moment!
Basics:
sub/sup/abbr
internationalization? [each role/directive has a different name]
Can generate various dicts to export
Admonitions
Math
amsmath, dollar, math directive
Figures
Referencing/numbering PR
numref/eq/ref
Working to remove all of the markdown-it-myst functionality –> docutils w/ review from ChrisS
How are we working with styles?
There is some copying back and forth of the .css
Damián
Learning Myst internals working in the Nikola + Myst story
Probably writing a series of blog posts to push discussion forward about things I have found and engage the community
MyST as a “standard” > What roles, directives, blocks do we need copied over Python (independent from Docutils/Sphinx)? Do we need a docultils/sphinx free Py representation of the “standard”?
Agenda items#
Let’s collect all potential agenda items here before the start of the meeting. We will then attempt to create a coherent agenda that fits in the 60m meeting slot. If there are similar items try and group them together.
NAME: ITEM (EXPECTED TIME TO DISCUSS)
Discuss team activity board proposal (executablebooks/meta#441)
The goal of this board is to coordinate and plan our major work items, to make it clear who is working on what, and to help signal boost items for review/discussion.
Is this board structure worth trying as-is or are there major changes to make to it?
If we want to try this process, perhaps we should use this meeting to add some initial items to the board?
Discussion:
This was discussed with the team and we agreed to use it this month to see how it works.
Any suggestions to simplify or improve the Activity Board was encouraged.
We focused on keeping the activity board pretty high level for cross-cutting communication and to help coordinate.
We also discussed keeping an eye on the
Request for Review
board to get visibility on PR’s that need review and get more distributed reviews across the team.We should add the Activity Board as a standing discussion point for the monthly meeting
Steve: Next Steps for Thebe
Refactoring the one big file into more manageable chunks
More tests - current test coverage is poor, PRs go green but not much really covered which is maybe worse than no tests at all
Kernel Status & Management - as part of refactor want to make it easier to get status of the kernel and lay ground work for the issues asking for: Kernel Status, Multiple Kernels and Jupyterlite connectivity; 412, 349, 271
Discussion:
This was discussed in the Reports section.
July 1st, 2021#
If you are joining the team video meeting, sign in below so we know who was here. Roll call:
Matthew McKay / Australian National University (ANU) / mmcky
Chris Sewell / EPFL / chrisjsewell
Steve Purves / Curvenote / stevejpurves
John Stachurski / ANU / jstac
Chris Holdgraf / 2i2c / choldgraf
Damián Avila / 2i2c / damianavila
Aakash Gude / ANU / AakashGfude
Reports, updates, and celebrations#
This is a place to make announcements (without a need for discussion), short updates about what you’ve been up to, and shout-outs to contributors! We’ll read through these at the beginning of the meeting.
QuantEcon / ANU#
QuantEcon: 4/5 Projects running Jupyter Book (remaining: julia)
ANU Team: Working on Support for Individual Page PDF (via LaTeX) generation (Lead: @mmcky)
Chris H notes that we should include updates about QuantEcon lectures conversion etc
@mmcky to add links to executable books gallery to QuantEcon jupyter-book powered projects
Chris S.#
Recent work
Released sphinx-external-toc and integrated in Jupyter-Book
Removes a lot of code from jupyter-book 😄, moving towards JB just being a thin wrapper for sphinx-extensions (last part is basically the
_config.yml
)Bit of a balancing act between; ease of use/understandanding, having a clear schema, having all the functionality of sphinx toctrees
Integration in JB also includes addition of “pluggable” CLI
Released markdown-it-py v1.0, i.e. fully stable
Released myst-parser v0.15.0, with sphinx v4 support
Should be moving all repositories to sphinx v4
Released rst-to-myst v0.3
Facile conversion of RST to MyST Markdown for a whole project
Utilises mdformat + mdformat-myst to create the Markdown from markdown-it tokens
Still working on mdformat-myst, but this essentially allows round trips Markdown text -> markdown-it tokens -> Markdown text. Could potentially be used for e.g. modifying links in downloadable notebooks
Created markdown-it-plugin-template; template repository with best practices for TypeScript library
Used this to create: markdown-it-amsmath, with demo website: https://executablebooks.github.io/markdown-it-amsmath/ and markdown-it-dollarmath, with demo website: https://executablebooks.github.io/markdown-it-dollarmath/
Intend to make all plugins for parity with mdit-py-plugins, and thus myst-parser extensions
Intended work
Update MyST VS Code extension to use new markdown-it plugins
Plus hopefully LSP support
Look into recently finalised Notebook API: https://code.visualstudio.com/api/extension-guides/notebook
Create sphinx-design: An iteration on sphinx-panels, but with the benefit of hindsight, and to more closely follow aspects of Materials Design and Material-UI
class names that do not clash with sphinx ones (i.e. container)
easier to set principle colors and width defaults
Either in myst-parser or sphinx-design have nice way to generate stylised post-header banners, with e.g. author, date, read time (see executablebooks/MyST-Parser#372)
Improvements to myst-nb (+ jupyter-cache)
Been gradually working on it, but lot of interconnected parts 😬
Revise the AST and transforms (post-transform -> transform)
Drop nbconvert, jupyter-sphinx dependencies
Better support for Markdown code output
Better support for “multi-level” configuration (cell level > notebook level > conf.py level)
Improve notebook execution and (maybe) drop
auto
mode in favour ofcache
Support for dynamic kernel names
How to handle glue; ideally in a kernel agnostic manner, possily with the new substitution syntax
Plugin CLI in jupyter-book for notebook execution
Steve#
Would like to:
meet the team
hear about thoughts on Thebelab as I’m prepping to do some work there
Berkeley team#
:wave: to Damian, onboarding etc
Damian is playing around with the MyST parser and learning how it is structured, experimenting with a Nikola implementation of the parser.
Chris is going to focus more of his time on team documentation, tool documentation for both users and developers, etc.
Agenda items#
sphinx-multitoc-numbering PR default behaviours (Lead: @mmcky)
Workshop for Theme Specification (Prerequisites?, Preparation?, Timing?) (Lead: @mmcky)
Any new features we should be focussing on from: https://executablebooks.org/en/latest/feature-vote.html
One possibility:
sphinx-margin
(executablebooks/meta#409)
Didn’t get to the following items
Bug issues in Jupyter Book: any important issues to address from: executablebooks/jupyter-book#issues
Jupyter-book in RTD: how can we make it happen!?
Chris H can give a short update from a conversation he had with Eric and Juan at RTD a few days ago.
Theming: I am still keen on creating a shared infrastructure (executablebooks/meta#279) and not necessarily being directly dependent on pydata-sphinx-theme
Although @choldgraf mentioned about possible funding from NumFocus for pydata-sphinx-theme
MyST as specification: what does that mean, how to “de-couple” from sphinx?
What roles/directives/extensions are “core”?
How to support additional roles/directives/extensions
How does this tie in with “JavaScript” work (i.e. editor tools in VS Code, JupyterLab, etc)?
Some notes in executablebooks/markdown-it-myst
John S:
Hiring casual RAs?
Chris H:
Communicating the work we’re doing - blog posts, tweets, etc? @executablebooks handle?
Coordinating around deliverables, workstreams, requests for reviews, etc?