tomminor

Building OpenEXR on Windows

I had issues getting a prebuilt version of OpenEXR working properly in my project so I decided to just build it myself, I ran into a couple of issues so I’ve quickly summarised them here incase anyone else has the same problem. In my case, I needed it to work on Windows 7 in Visual Studio 2013.

IlmBase Libs

IlmImf and other projects will complain about an error like this:

error MSB6006: "cmd.exe" exited with code -1073741515.

As explained in this Stack Overflow post, you have to manually copy the IlmBase libs (Half, IlmImf, Imath etc) into the OpenExr release/debug build directory  to successfully build.

Zlib unistd.h error

After I fixed that, I then got compile errors from within Zlib related to unistd.h being missing on Windows. Again, Stack Overflow saved my bacon and explained that you need to edit line 287 in zconf.h, changing

#if 1

to

#if HAVE_UNISTD_H

 

After this I managed to build it properly and use it in my project.

Telling an object not to cast shadows in Renderman RIS

by tomminor 0 Comments

I recently ran into an issue where I’d like to use a piece of geometry as a sky dome, seen through a glass window. This accidentally disabled my sun directional light, as it turns out the sky geometry was blocking it.

My usual fix was to add the “Indirect Lighting” attribute, but in this case it also prevented the rays refracted through the glass from reaching the sky dome geometry. After a bit of resarch I found https://renderman.pixar.com/view/TGT_REYES_Lighting, which explains Trace Sets as being the workaround to fix this.

The article already explains it well, but this is how I applied it in my case:

  1. Create a set with the geometry you’d like to not cast shadow
  2. In the attribute editor, make sure the set is selected and add the trace set attribute via Renderman >> Mark as Trace Group
  3. Now select the light you’d like to ignore this object, go to it’s Shadows tab and choose the trace set you just set up

trace_A

trace_B

Fractal Renderer is coming together

Fractal Renderer is coming together

I’ve spent the past few months developing a Optix based fractal renderer for my Major project team (as well as a little pipeline), it’s still unfinished but in the end I hope it has at least some of the following features (not all of them are done yet though):

  • Editable scene compiled at runtime into a hit function based on a node graph (the runtime compilation/ptx patching is done, but the node graph still needs work and I’ve fallen back to hard coding the scene description for now). Although the node approach likely won’t be 100% as controllable as writing all the clever scene description stuff by hand (like Shadertoy), it would allow me to quickly patch together a cool looking scene.
  • Passing in data from GUI controls without recompiling the whole hit function, maybe adding in the ability to keyframe values or derive their values per frame from Disney’s SeExpressions.
  • Path Traced Lighting (Done, but looks very odd on my fractal geo, possibly related to me not moving ‘out’ when I hit before firing a shadow ray)
  • Exporting to .exr with handy render passes (Done, OpenImageIO is great for this) – Here’s the output (my normal channel ends up a bit funky, likely related to the same issue that is causing the broken path tracing)

Screenshot from 2016-04-24 22:05:02

Current State

Although there’s still much to be done (outputting animating fractals for instance), this is the type of fractal I can currently render (with a little comp, thanks to the handy render passes):

kyranbulb2

 

Movement Test

The fractal in my major piece has to be moving, specifically some form of tunnel. Given the current state of the renderer, I had to do a separate test just to see how the fractals could move.

Procedural Klein Bottle in Python SOPS – Houdini

Procedural Klein Bottle in Python SOPS – Houdini

by tomminor 0 Comments

What’s a Klein Bottle?

Klein Bottle (3D)

A Klein Bottle can be thought of as a 3D Möbius strip, a surface with no interior that produces a shape that looks like an odd bottle. I don’t claim to be an expert at generating such things, but Paul Bourke is and you can read about it (and loads of other cool shapes) in more detail on his site.

Generating geometry in Houdini Python

While it seems possible to generate geometry in Vex, I decided to see how to do it with Python as it gives me an excuse to dabble in the Houdini Python API and I think it fits this type of project better. The bible for anything Houdini Python is the documentation itself, so I started looking at the examples there.

The main thing I discovered was that you can only generate geometry is limited to the Python SOP context, so a regular Python node will not work. You will need to make a new node using File >> New Operator Type >> Python Type, then set the Network Type to Geometry Operator.

Once all that’s set up it’s very easy to create geometry, simply use something like:

node = hou.pwd()
geo = node.geometry()
 
poly = geo.createPolygon()
 
p0 = geo.createPoint()
p0.setPosition( [0,1,0] )
p1 = geo.createPoint()
p1.setPosition( [1,1,0] )
p2 = geo.createPoint()
p2.setPosition( [1,0,0] )
p3 = geo.createPoint()
p3.setPosition( [0,0,0] )
 
poly.addVertex(p0)
poly.addVertex(p1)
poly.addVertex(p2)
poly.addVertex(p3)

Although I had to write some extra code to generate polygons properly, generating the points of the Klein was easy to do in a loop and pretty fast until I set it to high point counts.

Adding attributes to new points

The only other issue I had was adding attributes to points etc, I got errors related to the attribute not existing on the point. I eventually found an (obvious in hindsight) solution to this, you simply have to add the attribute to the geometry first and then you can set the value on the point:

nrmAttrib = geo.addAttrib(hou.attribType.Point, "N", (0.0, 0.0, 0.0))
 
for i in range(0, N):
    for j in range(0, N):
        u = umin + i * (umax - umin) / float(N)
        v = vmin + j * (vmax - vmin) / float(N)
        p = eval(u,v)
 
        p0 = geo.createPoint()
        p0.setPosition( p )
        p0.setAttribValue(nrmAttrib, calcNormal( eval(u, v), eval(u + delta, v), eval(u, v + delta)) )

The result

Here’s the final Klein geometry rendered out in Mantra with a glass shader, it’s missing a section because of the way I’m generating faces for it and I lack the time to debug it, but I think it makes it more unique than the generic klein bottle anyway.

klein

Bonus: Using it to test out my Reaction Diffusion (with a laplacian based on connected topology)

 

 

Venturing into OptiX – cudaMemcpy breaks everything

I’ve been looking into using Nvidia Optix for my Fractal lookdev tool, as I need realtime feedback while traversing the landscape.

I really like the API so far, it even comes with a Julia example demo in the SDK! I’ve modified it to render the Mandelbulb instead, with little trouble (and with refraction).

12318072_10208255826582355_1853094794_o

However, I needed to read the buffer data to write .exr’s to disk (this is a vital feature, better to make sure it’s viable at the beginning of the project). That shouldn’t be a problem, as I can easily grab the buffer’s device pointer and then cudaMemcpy it over.

All I need to do is include and , no big deal right?

#include <cuda.h>
#include <cuda_runtime.h>
 
#include <optixu/optixpp_namespace.h>
#include <optixu/optixu_math_namespace.h>
 
using namespace optix;
Linking CXX executable ../bin/julia
CMakeFiles/julia.dir/julia.cpp.o: In function `AnimCamera::apply(optix::Handle<optix::ContextObj>)':
/home/tom/src/optix/SDK/julia/julia.cpp:151: undefined reference to `PinholeCamera::PinholeCamera(float3, float3, float3, float, float, PinholeCamera::AspectRatioMode)'
/home/tom/src/optix/SDK/julia/julia.cpp:153: undefined reference to `PinholeCamera::getEyeUVW(float3&, float3&, float3&, float3&)'
collect2: error: ld returned 1 exit status

Oh.

Well maybe it’s because I included them before OptiX.

#include <optixu/optixpp_namespace.h>
#include <optixu/optixu_math_namespace.h>
 
#include <cuda.h>
#include <cuda_runtime.h>
 
using namespace optix;
/usr/local/cuda-7.0/include/cuda_runtime_api.h:257:17: error: ‘cudaError_t’ does not name a type
 extern __host__ cudaError_t CUDARTAPI cudaDeviceReset(void);
                 ^
/usr/local/cuda-7.0/include/cuda_runtime_api.h:274:36: error: ‘cudaError_t’ does not name a type
 extern __host__ __cudart_builtin__ cudaError_t CUDARTAPI cudaDeviceSynchronize(void);
                                    ^
/usr/local/cuda-7.0/include/cuda_runtime_api.h:349:17: error: ‘cudaError_t’ does not name a type
 extern __host__ cudaError_t CUDARTAPI cudaDeviceSetLimit(enum cudaLimit limit, size_t value);
                 ^
 
...
 
/home/tom/src/optix/SDK/julia/julia.cpp:299:95: error: ‘cudaMemcpy’ was not declared in this scope
   cudaMemcpy( (void*)h_ptr, (void*)d_ptr, sizeof(float) * totalPixels, cudaMemcpyDeviceToHost );

Oh.

This is dire, could it be related to the optix namespace somehow?

#include <optixu/optixpp_namespace.h>
#include <optixu/optixu_math_namespace.h>
 
using namespace optix;
 
#include <cuda.h>
#include <cuda_runtime.h>
Linking CXX executable ../bin/julia
[100%] Built target tutorial

Success! For some reason the optix namespace interferes with Cuda, setting it up like I did above somehow fixes that. Maybe this is mentioned in the documentation somewhere but I never found any reference to it.

So if you’re getting undefined references when you need cudaMemcpy in an OptiX project, check out your include order and namespace setup (and don’t forget to add ${CUDA_LIBRARIES} to target_link_libraries in the cmake configuration if it’s not there already).

I haven’t had any other strange problems besides this and so far OptiX seems pretty solid, I hope I’ll have fun with it.

Update

Turns out in this particular instance I was better off using the map() and unmap() functions within the Optix Buffer class instead of cudaMemcpy(), they remove the dependency of including the CUDA headers. However, I would no doubt have run into the same problems using something like the NVRTC headers.

Playing with Fractals in WebGL

Playing with Fractals in WebGL

As an excuse to play with WebGL I decided to finally got around to playing with the Mandelbrot and Julia set. There’s a live demo at the end of the post, the zoom control/mouse movement needs some work but the basic idea works great.

Links

A couple of useful places I used as reference :
Ozone3D – The Mandelbrot Set: Colors of Infinity
Fractals: Useful Beauty
dat.GUI – A great JavaScript GUI

The Shader Code

Vertex Shader

Really simple, all we need is a fullscreen quad with smooth UV’s.

attribute vec3 vtx_pos;
attribute vec2 vtx_uv;
 
varying vec2 texcoord;
 
void main(void)
{
	texcoord = vtx_uv;
	gl_Position = vec4(vtx_pos, 1.0);
}
Mandelbrot Set Fragment Shader

This one is very similar to the code in Ozone3D’s tutorial, I basically copied it for the most part (there’s only so many ways to skin a cat anyway).

The most notable aspect is MAX_ITER, which I replace with the actual iteration value before I compile the shader code. This was to get around the limitation of most webgl implementations which disallow you from using non-constant loop conditions, as they will be unrolled. This isn’t very convenient for real time display of any possible iteration size, so I simply recompile the shader when the iteration size changes.

Note that the “uniform vec2 c” uniform isn’t used here, it’s just there to keep the uniform layout identical for both shaders (I’m lazy and it’s just a demo).

Finally, I use a 1D texture (passed in through a 2D sampler, but the actual texture is only 1 pixel tall) to lookup the expected colour based on the iteration level.

#define iterations MAX_ITER
 
precision highp float;
 
varying vec2 texcoord;
 
uniform sampler2D gradientSampler;
uniform vec2 center;
uniform float zoom;
uniform vec2 c;
 
void main(void)
{
	// Mandelbrot
	vec2 C = vec2(1.0, 0.75) * (texcoord - vec2(0.5, 0.5)) * vec2(zoom, zoom) - center;
	vec2 z = C;
 
	gl_FragColor = vec4(0,0,0,1);
 
	for(float i = 0.0; i < float(iterations); i += 1.0)
        {
                z = vec2( z.x*z.x - z.y*z.y, 2.0 * z.x * z.y) + C;
                if(dot(z,z) > 4.0)
		{
			gl_FragColor = texture2D(gradientSampler, vec2(i / float(iterations), 0.5));
			break;
		}
	}
}

 

Julia Set Fragment Shader

The base code is identical to the Mandelbrot shader, except now we can pass in the desired complex parameter through c.

I can’t explain the maths as well as the pages I referenced earlier; but the main change is now the smooth UVs of the fullscreen quad are now assigned to Z instead of C, as the complex parameter is now a user defined constant and Z varies.

#define iterations MAX_ITER
 
precision highp float;
 
varying vec2 texcoord;
 
uniform sampler2D gradientSampler;
uniform vec2 center;
uniform float zoom;
uniform vec2 c;
 
void main(void)
{
	// Julia
	vec2 z = vec2(1.0, 0.75) * (texcoord - vec2(0.5, 0.5)) * vec2(zoom, zoom) - center;
 
	gl_FragColor = vec4(0,0,0,1);
 
	for(float i = 0.0; i < float(iterations); i += 1.0)
        {
                z = vec2( z.x*z.x - z.y*z.y, 2.0 * z.x * z.y) + c;
                if(dot(z,z) > 4.0)
		{
			gl_FragColor = texture2D(gradientSampler, vec2(i / float(iterations), 0.5));
			break;
		}
	}
}

The Live Demo (full page demo)

  • Toggle between the Mandelbrot/Julia set.
  • Use the zoom control to zoom in (this isn’t ideal).
  • Expect slow down on high iterations.
  • If you want to tinker around with the code then I apologise for how messy it is, it’s my very first adventure into JavaScript and I kept bolting more stuff onto it until I ended up with this.

ToDo

The Ozone3D page mentions that “a better approach is to use a multipass algorithm.”, which is next in line for this project.

It would also be cool to choose a complex parameter for the Julia set by clicking on the Mandelbrot itself, as both sets are connected.

Unreal Engine 4 on Linux – Problems with Optimus/Bumblebee “[VGL] ERROR: Could not open display :8”

The problem

Unreal Engine 4 works pretty well on the Linux desktop now, but I ran into unexpected problems when trying to run the editor natively on my NVIDIA Optimus equipped laptop. I tried to enable my dedicated gpu by using optirun but the “[VGL] ERROR: Could not open display :8”  error message was logged when I tried to load a project from the project browser.

This seems to be because optirun doesn’t like Unreal closing and forking a new editor process when the project is chosen. This is not a problem when opening a .uproject file directly, as nothing is closed and no new editor processes are forked. The problem (and potential solutions) are briefly described here, Bumblebee Documentation.

For most people the primus solution would probably work fine, but I never had any luck in getting primus to work so I had to figure out a tidy workaround for the alternative solution (opening a bash session with optirun and running the editor within that). 

A neat way of asking a new bash instance to run a bunch of commands is described here : http://stackoverflow.com/questions/7120426/invoke-bash-run-commands-inside-new-shell-then-give-control-back-to-user

My Solution

In the end I ended up using something like this to open a new optirun powered bash shell, start unreal and then wait until it exits before finally exiting the shell again.

# Where does unreal live
unreal_path=$HOME/src/UnrealEngine/Engine/Binaries/Linux
 
# Make a temporary file to store our commands in
tmp_file=$(mktemp)
 
# Open project window
echo "$unreal_path/UE4Editor >/dev/null 2>@1" >> $tmp_file
 
# Figure out the PID of the new editor instance that the project window spawned 
# (there are probably better ways of doing this, like pgrep)
 
# Tokenize the string
echo "editor_process=\$(ps -ejH | tail -n1)" >> $tmp_file
# We only want the first token from the string so we just grab the first item here
echo "editor_pid=( \$editor_process )" >> $tmp_file
 
# Wait until the editor closes
echo "while kill -0 \$editor_pid >/dev/null 2>@1; do sleep 1; done" >> $tmp_file
 
# Clean up and exit
echo "rm -f $tmp_file" >> $tmp_file
echo "exit" >> $tmp_file
 
# Launch a new optirun powered bash instance and ask it to execute the commands
# in the file
optirun bash --rcfile $tmp_file

My polished solution

Note that I use the fish shell, but the syntax should be simple enough to convert to bash if you feel inspired. Notable features include storing the log file for later reading if necessary, automatically figuring out if we should open a specific project file or the project window and fixing the project file path if it’s ambiguous.

function goUnreal
  set -l unreal_path $HOME/src/UnrealEngine/Engine/Binaries/Linux
  set -l tmp_file (mktemp)
 
  set -q UNREAL_LOG_FILE; and rm -f $UNREAL_LOG_FILE;
  set -U UNREAL_LOG_FILE (mktemp)
 
  echo "source ~/.bashrc" > $tmp_file
  echo "PS1=\"[Unreal Shell] ~> \"" > $tmp_file
 
  if count $argv[1] > /dev/null
    # Test if project file exists
    set -l project_file $argv[1]
    if test -e $project_file
      # Test if full path exists
      set project_file (pwd)/$project_file
      if test ! -e $project_file
        echo "Project file $project_file doesn't exist"
        return 1
      end
    end
 
    # Open specific uproject
    echo "$unreal_path/UE4Editor >$UNREAL_LOG_FILE \"$project_file\" 2>@1" >> $tmp_file
  else
    # Open project window
    echo "$unreal_path/UE4Editor >$UNREAL_LOG_FILE 2>@1" >> $tmp_file
 
    # Figure out the PID of the new editor instance that the project window spawned
 
    # Tokenize the string
    echo "editor_process=\$(ps -ejH | tail -n1)" >> $tmp_file
    # We only want the first token from the string so we just grab the first item here
    echo "editor_pid=( \$editor_process )" >> $tmp_file
 
    # Wait until the editor closes
    echo "while kill -0 \$editor_pid >/dev/null 2>@1; do sleep 1; done" >> $tmp_file
  end
 
  # Clean up and exit
  echo "rm -f $tmp_file" >> $tmp_file
  echo "exit" >> $tmp_file
 
  optirun bash --rcfile $tmp_file
end

 

Copy blendshape to similiar geometry script (Maya)

A quick script I wrote that helps you copy blendshapes from a target mesh to only the selected vertices of another mesh, which is useful if you have models with identical faces but different bodies (that are a single object).

For example, if you want to copy a face blendshape from a character onto a copy of the character with identical face geometry but different body topology, you would :

  1. Duplicate the character with different body topology.
  2. Select the original blendshape mesh that you want to transfer to the duplicate.
  3. Shift select the vertices on the duplicate character that you want to be affected (such as all the head vertices).
  4. Run my script.
  5. Blendshape >> Add the result to your skinned character.

I’m unsure if there is a better way of doing this, but this script worked for a simple cartoon character.

import maya.cmds as cmds
 
selection = cmds.ls(sl=True)
 
if len(selection) > 0:
    targetMesh = selection[-1]
    affectedVertices = selection[0:-1] # Assume the rest of the selection is made of vertices
    affectedMesh = cmds.listRelatives(cmds.listRelatives(affectedVertices[0], p=True), p=True)
 
    vtxSet = cmds.sets(affectedVertices, v=True, n="SelectionSet")
    print cmds.sets(vtxSet, q=True)
 
    # Move to origin
    cmds.move(32, 0, 0, affectedMesh, ws=True)
    cmds.move(32, 0, 0, targetMesh, ws=True)
 
    # Select target mesh and affected vertices
    cmds.select(cl=True)
    cmds.select(targetMesh)
    cmds.select(vtxSet, add=True)
 
    cmds.transferAttributes(transferPositions=1, transferNormals=1, transferUVs=2, transferColors=0, sourceUvSpace="map1", sampleSpace=3, searchMethod=0, flipUVs=0, colorBorders=1)
 
    cmds.hide(targetMesh)
 
    cmds.select(cl=True)
else:
    print "Select vertices to move and target mesh"

 

Linux Bash/Terminal – Advanced – Pattern Searching

Bash Pattern Searching Reference (PAL Series)

Update: These were written for my students while my role was a PAL (Peer Assisted Learning) Leader for my course, I intended to keep all the basic/intermediate commands necessary for terminal use in one place to ease the students into using Linux efficiently. I’ve kept them up just in case they’re of use to someone.

Wildcards

* – Match anything
? – Match a single character

$ ls
render_10.exr render_13.exr scene_10.tif scene_9.exr
render_11.exr render_14.exr scene_11.exr scene_9.tif
render_12.exr scene_10.exr scene_11.tif
$ ls scene*       # Match any file beginning with "scene"
scene_10.exr scene_11.exr scene_9.exr
scene_10.tif scene_11.tif scene_9.tif
$ ls *.exr        # Match any file ending with ".exr"
render_10.exr render_12.exr render_14.exr scene_11.exr
render_11.exr render_13.exr scene_10.exr scene_9.exr
$ ls scene_?.tif  # Match scene_1.tif, scene_2.tif, scene_3.tif etc but not scene_10.tif
scene_9.tif
$ ls scene_??.tif # Match scene_10.tif etc but not scene_1.tif etc
scene_10.tif scene_11.tif

Grep

Allows for much more powerful pattern matching than wildcards alone, but you need to learn regular expressions to really utilise it’s power.

A simple usage however is to search the output of a command

$ history | grep "touch *"
 2006 touch output.exr 
 2011 touch {a,b}{x,y,z}
 2013 touch {a,b}/{x,y,z}
 2021 touch {a,b}/{x,y,z}
 2022 touch {a,b}{x,y,z}

You can also use it directly on a file to search just that file

$ grep TARGET tracer.pro      # Search for string "TARGET" in tracer.pro 
TARGET=tracer
 QMAKE_EXTRA_TARGETS += first copydata
 PRE_TARGETDEPS+=C:/NGL/lib/NGL.lib

Linux Bash/Terminal – Advanced – Expansion (Time saving tricks)

Time Saving Tricks Reference (PAL Series)

Update: These were written for my students while my role was a PAL (Peer Assisted Learning) Leader for my course, I intended to keep all the basic/intermediate commands necessary for terminal use in one place to ease the students into using Linux efficiently. I’ve kept them up just in case they’re of use to someone.

Brace Expansion

Expanding two sets of braces

$ touch {a,b}{x,y,z}
$ ls
ax ay az bx by bz

Nesting expanding braces with existing directories

$ mkdir include src
$ touch include/{vector,matrix,complex}.h src/{vector,matrix,complex,main}.c
$ ls
include src
$ ls include/
complex.h matrix.h vector.h
$ ls src/
complex.c matrix.c main.c vector.c

As you can see, brace expansions work like expanding the brackets in algebra.
They’re very handy for quickly generating directory structures with little typing. Manual page.

Sequence Expansion

Create 16 files

$ touch file_{0..16}.txt # Sequence from 0 to 16
$ ls
file_0.txt file_12.txt file_15.txt file_2.txt file_5.txt file_8.txt
file_10.txt file_13.txt file_16.txt file_3.txt file_6.txt file_9.txt
file_11.txt file_14.txt file_1.txt file_4.txt file_7.txt

Create directories with odd numbers only in their names

$ mkdir dir_{1..9..2}     # Sequence from 1 to 9, with a step of 2 (skip every other number)
$ ls
dir_1 dir_3 dir_5 dir_7 dir_9