Search Blogs

Tuesday, August 22, 2023

Is Pluto.jl loosing its luster

Is Pluto.jl loosing its allure as a notebook environment for computational scientist? Why would I ask such question? Because for the past 6 months it seems that the Pluto.jl developers, for which I can tell there are only a few, have been devoting all their time to Pluto.jl as a teaching platform. This isn't bad at all and is a good use case for Pluto. The problem is that in my opinion there are so many other issues and needed features that should be worked on if the intent of Pluto.jl is to replace IJulia as the defacto notebook environment for Julia; which has always been my hope.

What is so good about Pluto notebooks

Pluto.jl is a notebook environment specifically for Julia and it has the special feature that it is entirely reactive. This means that Pluto notebooks track the connectivity of the cells via variable dependencies. The example from the github repo README is shown below. If your coming from Jupyter notebooks, you will either hate this or love it. At first, I loved it, because I have always hated having to rerun a bunch of cells in Jupyter notebook to get everything up-to-date. The downside is that if you have a cell that takes a good amount of time to compute (i.e. training a neural network) you can end up getting frustrated because you changed a cell with a variable that the training loop depends on. The work around for this isn't too hard, you can make use of PlutoUI.jl utilities so that the cells with a heavy compute doesn't run till its triggered by a HTML widget binding.

Showing the reactivity of cells in a Pluto.jl notebook

Pluto also focuses on simplicity and clarity, which I love. I think the notebooks look clean and elegant. Markdown cells are the same as compute cells but you trigger markdown by pressing CTRL+M which gives you a cell that is pre-populated with the string macro md""" """. You can then use any standard Markdown you want and press SHIFT+Enter to render the cell to markdown. You can easily rearrange the visual appearance of the cells in a notebook and don't have to worry about flow dependencies (this is what reactivity handles for you!).

One really nice feature of pluto is that the notebooks are just pure Julia code with comment mark-ups, this means that as long as you have the project environment setup you can run a Pluto notebook from the command line. I made a little utility to help with setting up a Julia project environment from a Pluto notebook. Initial, you had to configure the package environment yourself but since v0.15 you don't have to any more, all you need is to define a cell as such:

begin
    using Plots
    using PlutoUI
end

and the built in Pluto package manager will handle everything 1. When you want to share a Pluto notebook you can do it in several ways. You can just share the .jl notebook file and they can run in on their own Pluto server instance. You can give them a link to the .jl notebook file and they can open the link directly from the Pluto server instance. My favorite is to export the notebooks to HTML because they can be directly read in a browser and you don't loose computability because every HTML pluto notebook can be downloaded/saved as a .jl,

HTML version of Pluto notebook lets you save .jl notebook file!

This is super useful when you want to make your work accessible to anyone regardless of if they have Julia/Pluto setup. Futhermore it makes it easy to integrate your notebooks into a website or blog.

For me Pluto really seems (or should I say seemed) like the ideal computing environment to do data analysis, plotting, and draft scientific2 writing. I think its superior to Jupyter notebooks and if something like Google Colab. was available for Pluto, it would be the defacto notebook environment for Julia.

The problems

Notice

These are just my observations and opinions. I don't know what is actually the intent and focus of the Pluto developers, so I could be getting everything wrong here.

In the past 6 months I've noticed a shift of focus from the developers on the core features of Pluto.jl. The primary focus seems to be on using Pluto as a teaching environment for computing. While this is undoubtedly cool and useful, it leaves all other badly need aspects behind.

So what are some of this issues I think need to be addressed? Well here they are:

  1. Open a notebook without running the cells. Sure I can just look at the .jl notebook file and see whats going on, but this is obnoxious because sometimes I just want to view things in Pluto!
  2. Ability to set templates. I have notebook I typically use as the starting template for my work, but I have to copy and rename it from the command line and then open it in Pluto. Why not just have this configured in the Pluto server?
  3. Support for briefcase or portfolio. Usually I'll have multiple notebooks that are related but focus on a very specific analysis or computation, but they typically use the same packages and need to be cross-referenced (this is especially the case when considering using them in a book or website). Why not make use of this and prevent each notebook from loading a new project environment by creating a portfolio project which also has the support for cross referencing HTML ids? This is badly needed in my opinion. Surely having a single environment would be faster. Eventually we want something like Jupyter books. "Tabing" across notebooks would be nice too!
  4. Download notebook to julia project. This should do what my simple utility does but better.
  5. More widget types in PlutoUI.jl. This is a powerful utility package for Pluto.jl because it alleviates users having to implement HTML or CSS. The problem is there are so many more to be done. For example I made a widget that creates a table of packages and their versions so people who view the HTML version know which versions are used.
  6. Polish the HTML5 recording option. This is awesome, but seems like there is a good amount of improvement needed.

There are a lot of other request in the Github issues section that are good, but these are just mine. I also have some less critical interest such as including a built in LLM similar to the live docs. I put together a in-line LLM chat called Hermes.jl which isn't bad, but something like live docs would be better, I think. Then if the Quarto and Pluto teams could coordinate so that Pluto server and notebooks can be used that would be pretty cool, although still need to figure out the cross-referencing.

What to do

So what's the point of all this. Unfortunately, the big push toward "Pluto as a teaching framework" forces me to be cautious in future use. I want to use Pluto exclusively for all my data analysis and computing write-ups, but because it lacks the features I eventually am looking for, I will eventually need to look to something else. At the moment its looking like Quarto will be the go to for me. Maybe I'm wrong and the whole Pluto teaching website template is the best path forward, I just don't see it.

Footnotes


  1. By default the built in Pluto package manager grabs the most recent version of the package. If you want a specific version then you need to disable the built in package manager by using the old way

  2. Pluto isn't quite complete enough to enable writing a scientific manuscript, but I could envision it being possible. One major need would be baked-in support for cross-referencing in a pandoc or $\LaTeX$ style. 


Reuse and Attribution