mayavi 4.8.3 installation bugs - part 2
In the previous post I was attempting to install mayavi 4.8.3 and ran into a bunch of issues. This process didn't good smoothly on Ubuntu 2404 and I had to spend multiple hours debugging.
I finally got the package installed and thought things would be good so I called it a day. Now its a new day, but no, it still doesn't work when you try to run it.
Here's a very simple python file that you would hope would work properly if you have installed a package:
import mayavi.mlab as mlab
Attempting to run this very simple file:
$ uv run mayavi_test.py
Traceback (most recent call last):
File "/home/janis/experimental-code/mayavi_test.py", line 3, in <module>
import mayavi.mlab as mlab
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/mlab.py", line 15, in <module>
from mayavi.core.common import process_ui_events
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/core/common.py", line 16, in <module>
from apptools.persistence.state_pickler import create_instance
ModuleNotFoundError: No module named 'apptools'
As you can see the default installation of mayavi doesn't work. I went to the docs again to see if there was some install step that I was missing. Apparently on Python 3.x you have to use PyQt5 but that isn't automatically installed as a dependency. https://docs.enthought.com/mayavi/mayavi/installation.html
So following this I installed PyQt5 using
$ uv add PyQt5
And attempted to re run:
$ uv run mayavi_test.py
Traceback (most recent call last):
File "/home/janis/experimental-code/mayavi_test.py", line 3, in <module>
import mayavi.mlab as mlab
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/mlab.py", line 15, in <module>
from mayavi.core.common import process_ui_events
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/core/common.py", line 16, in <module>
from apptools.persistence.state_pickler import create_instance
ModuleNotFoundError: No module named 'apptools'
Note that I installed mayavi first then PyQt5 second. Doing some more digging it appears that apptools is a package made by enthought that is requires by mayavi but is not tracked in mayavi's dependencies when mayavi is installed by uv. Details on PyPi here: https://pypi.org/project/apptools/. Not tracking this dependency strikes me as poor package management. If this was optional adding some more detail here would be nice for end users. That could be as simple as:
try:
import apptools
except ImportError:
print("if you want to use this module you need to install the apptools dependency first")
raise
But if this dependency is not optional this is just a packaging bug.
So I installed this dependency using uv:
$ uv add apptools
Resolved 27 packages in 159ms
Installed 2 packages in 15ms
+ apptools==5.3.1
+ traits==7.0.2
Attempting to run again:
$ uv run mayavi_test.py
Traceback (most recent call last):
File "/home/janis/experimental-code/mayavi_test.py", line 3, in <module>
import mayavi.mlab as mlab
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/mlab.py", line 15, in <module>
from mayavi.core.common import process_ui_events
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/core/common.py", line 21, in <module>
from pyface import api as pyface
ModuleNotFoundError: No module named 'pyface'
Another untracked dependency... :/
This time pyface: https://pypi.org/project/pyface/
$ uv add pyface
Resolved 28 packages in 133ms
Installed 1 package in 43ms
+ pyface==8.0.0
Attempting to run again:
$ uv run mayavi_test.py
Traceback (most recent call last):
File "/home/janis/experimental-code/mayavi_test.py", line 3, in <module>
import mayavi.mlab as mlab
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/mlab.py", line 16, in <module>
from mayavi.tools.camera import view, roll, yaw, pitch, move
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/tools/camera.py", line 24, in <module>
from .engine_manager import get_engine
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/tools/engine_manager.py", line 13, in <module>
from mayavi.preferences.api import preference_manager
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/preferences/api.py", line 4, in <module>
from .preference_manager import preference_manager
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/preferences/preference_manager.py", line 29, in <module>
from traitsui.api import View, Group, Item
ModuleNotFoundError: No module named 'traitsui'
Another untracked dependency... :/
This time traitsui: https://pypi.org/project/traitsui/
$ uv add traitsui
Resolved 29 packages in 273ms
Installed 1 package in 24ms
+ traitsui==8.0.0
Attempting to run again:
$ uv run mayavi_test.py
Traceback (most recent call last):
File "/home/janis/experimental-code/mayavi_test.py", line 3, in <module>
import mayavi.mlab as mlab
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/mlab.py", line 16, in <module>
from mayavi.tools.camera import view, roll, yaw, pitch, move
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/tools/camera.py", line 24, in <module>
from .engine_manager import get_engine
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/tools/engine_manager.py", line 13, in <module>
from mayavi.preferences.api import preference_manager
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/preferences/api.py", line 4, in <module>
from .preference_manager import preference_manager
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/preferences/preference_manager.py", line 127, in <module>
preference_manager = PreferenceManager()
^^^^^^^^^^^^^^^^^^^
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/preferences/preference_manager.py", line 81, in __init__
self._load_preferences()
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/preferences/preference_manager.py", line 111, in _load_preferences
default = preferences.node('default/')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/apptools/preferences/scoped_preferences.py", line 221, in node
node = self._get_scope(scope_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/apptools/preferences/scoped_preferences.py", line 406, in _get_scope
scope = self.get_scope(scope_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/apptools/preferences/scoped_preferences.py", line 373, in get_scope
for scope in self.scopes:
^^^^^^^^^^^
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/apptools/preferences/scoped_preferences.py", line 357, in _scopes_default
Preferences(
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/apptools/preferences/preferences.py", line 91, in __init__
self.load()
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/apptools/preferences/preferences.py", line 358, in load
from configobj import ConfigObj
ModuleNotFoundError: No module named 'configobj'
Another untracked dependency... :/
This time configobj: https://pypi.org/project/configobj/
$ uv add configobj
Resolved 30 packages in 392ms
Installed 1 package in 4ms
+ configobj==5.0.9
Attempting to run again:
$ uv run mayavi_test.py
Traceback (most recent call last):
File "/home/janis/experimental-code/mayavi_test.py", line 3, in <module>
import mayavi.mlab as mlab
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/mlab.py", line 16, in <module>
from mayavi.tools.camera import view, roll, yaw, pitch, move
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/tools/camera.py", line 24, in <module>
from .engine_manager import get_engine
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/tools/engine_manager.py", line 15, in <module>
from mayavi.core.engine import Engine
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/core/engine.py", line 33, in <module>
from mayavi.core.ui.mayavi_scene import viewer_factory
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/mayavi/core/ui/mayavi_scene.py", line 12, in <module>
from tvtk.tools.ivtk import IVTK
File "/home/janis/experimental-code/.venv/lib/python3.12/site-packages/tvtk/tools/ivtk.py", line 32, in <module>
from pyface.api import FileDialog, GUI, OK, PythonShell
ImportError: cannot import name 'PythonShell' from 'pyface.api' (/home/janis/experimental-code/.venv/lib/python3.12/site-packages/pyface/api.py). Did you mean: 'IPythonShell'?
WTF is going on here, talk about sloppy package management taken to an extreme. While the error message may look slightly different this is much the same as the other errors, the package depends on ipython but that wasn't tracked by the other packages and hence wasn't installed along with the initial install step. Explicitly adding ipython as a dependency in:
$ uv add ipython
Resolved 46 packages in 449ms
Prepared 1 package in 600ms
Installed 16 packages in 133ms
+ asttokens==3.0.0
+ decorator==5.2.1
+ executing==2.2.0
+ ipython==9.4.0
+ ipython-pygments-lexers==1.1.1
+ jedi==0.19.2
+ matplotlib-inline==0.1.7
+ parso==0.8.4
+ pexpect==4.9.0
+ prompt-toolkit==3.0.51
+ ptyprocess==0.7.0
+ pure-eval==0.2.3
+ pygments==2.19.2
+ stack-data==0.6.3
+ traitlets==5.14.3
+ wcwidth==0.2.13
Running again:
$ uv run mayavi_test.py
Success!
What really shits me about all this is that if the only way to get these things installed properly is via edm (which is the enthought python environment manager) then the docs should just say this. Saying you can just install via pip install seems wildly inaccurate. I've been exceedingly stubborn here trying to install this but the average user will likely just give up and go use something else. And rightfully so.
As an aside this situation is an example of where a lockfile starts to be very valuable, because I've had to install a lot of other packages that really should have been tracked by the dependencies of the main package install I now want to be very sure I can freeze in time this exact combination of the dependencies as they are now. I simply don't trust that I can just run the command pip install mayavi or uv add mayavi to install again because the package dependencies for these packages aren't being properly tracked. This is a situation where if I get the combination of installed packages exactly right I don't want to mess with them again, especially the transitive dependencies since those don't seem to be properly managed here.
This post is part 2 of the "mayavi-installation" series: