VtkNodes: Using the Visualization Toolkit inside Blender

authors:  Silvano Imboden, Paul McManus, Lorenzo Celli

VTKNodes is a Blender Addon providing the possibility to use VTK within Blender

The Visualization Toolkit (VTK https://www.vtk.org/ ) is an open source library for scientific visualization widely

adopted by researcher in fields such as geo-physics, fluid dynamic and biomedicine. VTK is developed

by Kitware, which is also the author of CMake, the build configurator used by Blender.

3DSlicer and Paraview ( https://www.kitware.com/platforms ) are some of the open source applications built on top of VTK.

Some example of Scientific Visualizations done with VTK:

In VTK the process of reading/transform/display data is accomplished by instantiating and connecting

among them a series of reader/filters/actors so it naturally fits the metaphor provided by the Blender Node Editor, but the implementation of a VTKNodes addon for Blender was not feasible until last year, when Kitware

finally provided the VTK bindings for Python3.x.


My personal interest in using VTK from the Blender user interface is aimed at prototyping VTK visualization networks and also to ease teaching and learning VTK. Beside my personal interest , VTK is lacking by design any editing tools, in the past people using VTK were implementing custom editing tools, but recently I saw an increasing number of people preferring to move their data in and out Blender to use the  tools that Blender provides. Therefore I hope that VTKNodes may be helpful in creating new visualization-applications entirely within Blender.

Figure 1: An a simple example of a VTKNodes network that loads a CTScan data, extract two different Isosurfaces, and transform them to a  Blender mesh.

Project Status at 27 July 2018

VTKNodes is not completed yet, but the work done so far is very promising.

At this moment we provided nodes for 741 classes out of 900.

We plan to release  release a first version before the next Blender conference,

History of Development

the rest of this document provides full details on:

  1. Compile a VTK compatible with Blender
  2. Using VTK inside Blender first approach:  let VTK do the drawing
  3. Transforming a vtkPolyData to a BlenderMesh and Vice Versa
  4. Writing an addon providing custom nodes
  5. Decide which VTK classes should be represented as Blender Nodes
  6. Automatic code generation, customizing the generated code
  7. Further node customization
  8. User driven properties exclusion
  9. VTK Class documentation
  10. VTK Class inspection
  11. Transferring VTK generated colors as Blender Vertex Data
  12. Examples



Silvano Imboden   s.imboden@cineca.it

Compile a VTK compatible with Blender

My last goal will be to provide libraries ready to be used for the most common OS, for other platforms I may provide detailed information on how to compile your own version.

These are the steps that worked for me with VTK8 and Blender 2.79 on Ubuntu 16.04 LTS.
In short you have to compile
using a Python version that matches  the one in Blender, then compile VTK and its python bindings using the same version of  Python, lastly you need to configure the environment so that the various libraries can be found from Blender.

find out which python version that blender is using

run blender, create a py-console,

>>> import sys

>>> sys.version

'3.5.3 (default, May 18 2017, 14:40:48) \n[GCC 5.4.1 20161019]'

download and compile python
> wget https://www.python.org/ftp/python/3.5.3/Python-3.5.3.tgz

> tar -xvzf Python-3.5.3.tgz

> cd python-sources

> ./configure --prefix=../install --with-computed-gotos --with-pymalloc [--enable-shared or --enable_framework on osx ]

the last three options should build python matching the way it is built for blender

> make -j 4

> make install

> cd ../install/bin

> python

Python 3.5.3 (default, Nov  3 2017, 17:31:29)

[GCC 5.4.0 20160609] on linux

Type "help", "copyright", "credits" or "license" for more information.

download and compile vtk using the previous python

prepare the following folders:

xxx/VTK8-py353/src -- explode the vtk source here



run cmake-gui, and set src and build dir, press generate

set CMAKE_BUILD_TYPE = Release

set INSTALL_PREFIX= xxx/VTK8-py353/install

set module_vtkImagingOpenGL2 = on

set module_vtkPython = on

set module_vtkWrappingPythonCore= on

press generate

set PYTHON/PYTHON_INCLUDE_DIR = your_python_install_dir/include

set PYTHON/PYTHON_LIB_DIR =  your_python_install_dir/lib



> cp xxx/PY353/include/python3.5m/patchlevel.h  xxx/PY353/install/include/patchlevel.h

press generate

> cd ../build

> make -j 8

> export PYTHONPATH= xxx/PY353/install/lib/python3.5:$PYTHONPATH

> export LD_LIBRARY_PATH= xxx/PY353/install/lib:$LD_LIBRARY_PATH

> make -j 8

> make install

configure the environment

> export PYTHONPATH= xxx/VTK/install/site-packages:$PYTHONPATH

> export LD_LIBRARY_PATH= xxx/VTK/install/lib:$LD_LIBRARY_PATH

test it

Run Blender, open the python console

> import vtk

> dir(vtk)

[ ‘DC’… ….. 'vtkZLibDataCompressor' ]

Using VTK inside Blender, Low Level mode: let VTK draw on Blender

It is possible to have VTK drawing inside a Blender Viewport, instead or togetherwith Blender.
It is possible to have the VTKCamera synchronized with the Blender Camera.

My purpose for this was mostly to debug VTKNodes but it may be useful to provide some exotic feature of VTK like volume rendering. This feature is provided by a small Addon linked here which adds a VTKPanel to the 3DView  with two checkboxes to turn on and off the VTK rendering in exclusive or overlayed mode.

Figure 2: the cone is drawn by VTK, the Cube is drawn by Blender, if you rotate the camera both objects rotate, if you move or resize the view everything works as expected. To the side there is the code that does the work.

This works with VTK OpenGL system 1.0 only ---- on Blender 2.8 we may be able to use a more recent version of OpenGL 

Figure 3: some of the vtkData types

Figure 4: some of the most exotic VTK Source nodes

Transforming a vtkPolyData to a Blender Mesh and vice versa

In VTK, apart from when you are using Volume Rendering, all types of data must be transformed to a vtkPolyData before being visualized. Transforming a vtkPolyData without attributes to a Blender Mesh is verysimple.

vtkData Attributes can be mappedto Vertex Data ( see later on )

The opposite conversion ( of static blender mesh ) is very straightforward.

Find out how to do a Blender Addon providing Custom Nodes

Blender provide a template for custom nodes which is a good starting point, the few other things I needed I learned looking inside the “Animation Nodes”.

The unusual thing for me was that nodes are not object like you expect in Object Oriented Programming,
you can’t store or attach anything inside a Blender Node. The reason for this is that Blender provide you the possibility to define your custom kind of nodes, instantiate and connect them, and have your network saved and later restored from a Blender file, Blender will not being able to serialize your
 node network if it contains data defined outside the normal Blender data structure. Thus one way to execute a node-network is to read all the nodes just in time, understand what is needed to do and then do it, repeating this each time.

This approach is not the best for the VTKNodes, because we don’t want to recreate the whole VTK pipeline each time, re-executing any Reader and so on. So we need a cache system to store the underlying vtk-objects

once and then reuse them and the data inside them every time the user interacts with the node-network.

I ended-up creating a map local to any VtkNode network, indexed on the vtkNode names and containing its corresponding vtk-object. The map is populated when it is discovered that it is empty ( for example if you reload the addon code, the previous data will be lost, and there are no events that let you know about it )

As shown in figure 1, in a VTKNodes network the termination nodes have an “Update” button.
When you press the update button for the first time, all the incoming nodes are visited, their vtk-object are created and cached. When you press it a second time, all incoming nodes are visited, for each node property a matching method on the vtk-object is called, and for each node connection a
correspondingconnection is made among the vtk-objects. In the end all the vtk-object are executed and their output is transformed to a Blender Mesh.

ensures that when the user changes anything in the network, only the affected parts will be re-executed.

Decide which VTK classes should be represented as Blender Nodes

VTK provides more then 1000 classes, some of them are internal components and are not meant to be instantiated by the user, some are virtual and are not meant to be instantiated at all.

My first approach was to consider the following categories: Readers, Writers, Sources and Filters.

In the past finding these classes was straightforward, now all of these inherit from vtkAlgorithm, and you have to check at runtime if a class has inputs ( filters, writers, and other sinks ) or if it has outputs ( filter, readers,…  )

Now I realized that I’ll need more classes, which are not strictly vtk-pipeline-elements but are used as pipeline parameters. Examples are ColorMaps, Integrators, ImplicitFunctions and so on. I haven't implemented support for these yet.

Nodes code generation

VTK is huge, therefore manually coding each node is not an option. Code for each node will be generated by some python script.

So I created a “fat”  VTKNode-base-class and stored there most of the code needed : creating the object, synchronizing the vtk-object internal status with the corresponding Node properties, synchronizing vtk-object connections with node connections, executing the network.

Having a “fat” base class, simplify the coding needed in derived classes,  which in the end only need to declare their properties ( name, type and default value of each property ).[a]  the next figure shows the generated source code for the the vtkConeSource node class.


The next figure shows how the code is organized.

At the bottom there is the mentioned NodeBaseClass, the Yellow layer represent the generated code for the specific classes organized in modules by categories ( gen_sources, … ).  These modules are imported by a second layer of modules ( sources, … ) that provide the possibility of overriding/customizing any of the generated classes if/when this is required. Finally, all the previous modules are imported by the Addon Main File which performs the nodes registrations in Blender.

The code generation exploits the strict method naming convention followed by VTK and the Python introspection features: A first python script enumerates the vtk classes and tries to instantiate them in turn, virtual classes not meant to be instantiated will raise an exception, classes inheriting vtkAlgorthm will be classified as sources, reader, writer, …. the other classes are ignored. For the interesting classes we now enumerate the methods and we try to match methods in pair as follows

If I find a method named SetXXX, I search for another method named GetXXX, if I find it I look at the method documentation which reports the type of the arguments, if the arguments match

( i.e.  I found void SetXXX( int ) and  int GetXXX() ) then we know that this object has a property named ‘XXX’

with type int and default value = GetXXX().

For int properties meant to be a boolean VTK provide also the method void SetXXXOn(),  void SetXXXOff()

For int properties meant to be a combobox VTK also provides the methods

void SetXXXToYYY(), void SetXXXtoZZZ(), void SetXXXToWWW()

I am also using some heuristics to hide methods not meant to be exposed to node users, like Incr/DecrRefCount.

All the information is stored in a sqlite database that helps me to count and track properties analysis failures. Many of the failures are false alarms ( like finding a methods  SetX( int, int ) when you also have SetX( int[2])  and int[2] GetX()   ) other are genuine failures. I identified 10 types of weird signatures, and for the moment I can address most of them by manually override the generated code.

A second python scripts reads the information from the database and creates the specific node source code using a Jinja2 template system.

User driven properties exclusion

When VTKNodes processes a node network, the setter of each property will be called on each vtk-object.

Some methods are intended to be used in mutual exclusion, for example you can provide the vtkConeSource
with radius and height, or with radius and angle, but if you set all the three what will happen depends on the order of the calls. In these case it is better to forget one of the three properties, but which ?

The current code provide the user the possibility to show and hide any properties of each node type. These settings are stored Blender-wide so they will be preserved among different project or different Blender files.

This feature and the following were implemented by Lorenzo Celli.

VTK class documentation

for each vtkNode it is possibile to read the documentation of the corresponding vtk-class

VTK class inspection

VTKNodes provides features to inspect the status of each node, and of each node output.

Transferring VTK generated colors as Blender Vertex Data

Colors are important in Scientific Visualization.

Any numeric attribute of a VTK Data can be mapped to colors.

With the limitation of exporting a single user-chosen numeric attribute, the vtk_to_blender_node now stores the mesh attributes as vertex data, and it is possible possible to create a ColorTable node to control how values are mapped to colors.

an example showing attributes created by an elevation filter on a sphere

example of colored streamtubes ( the kitchenflow vtk-example recreated in blender )


Examples can obviously be shipped as blender files, but we were looking for something lighter so that we can distribute it within the addon, so Lorenzo provided an example machine: a network can be stored as a json file inside the addon folder, examples can be loaded from a menu in the user interface, examples can be organized in sections. Now we just need to make the examples :-)