The problem

Unreal Engine 4 works pretty well on the Linux desktop now, but I ran into unexpected problems when trying to run the editor natively on my NVIDIA Optimus equipped laptop. I tried to enable my dedicated gpu by using optirun but the “[VGL] ERROR: Could not open display :8”  error message was logged when I tried to load a project from the project browser.

This seems to be because optirun doesn’t like Unreal closing and forking a new editor process when the project is chosen. This is not a problem when opening a .uproject file directly, as nothing is closed and no new editor processes are forked. The problem (and potential solutions) are briefly described here, Bumblebee Documentation.

For most people the primus solution would probably work fine, but I never had any luck in getting primus to work so I had to figure out a tidy workaround for the alternative solution (opening a bash session with optirun and running the editor within that). 

A neat way of asking a new bash instance to run a bunch of commands is described here : http://stackoverflow.com/questions/7120426/invoke-bash-run-commands-inside-new-shell-then-give-control-back-to-user

My Solution

In the end I ended up using something like this to open a new optirun powered bash shell, start unreal and then wait until it exits before finally exiting the shell again.

# Where does unreal live
unreal_path=$HOME/src/UnrealEngine/Engine/Binaries/Linux

# Make a temporary file to store our commands in
tmp_file=$(mktemp)

# Open project window
echo "$unreal_path/UE4Editor >/dev/null 2>@1" >> $tmp_file

# Figure out the PID of the new editor instance that the project window spawned 
# (there are probably better ways of doing this, like pgrep)

# Tokenize the string
echo "editor_process=\$(ps -ejH | tail -n1)" >> $tmp_file
# We only want the first token from the string so we just grab the first item here
echo "editor_pid=( \$editor_process )" >> $tmp_file
    
# Wait until the editor closes
echo "while kill -0 \$editor_pid >/dev/null 2>@1; do sleep 1; done" >> $tmp_file

# Clean up and exit
echo "rm -f $tmp_file" >> $tmp_file
echo "exit" >> $tmp_file

# Launch a new optirun powered bash instance and ask it to execute the commands
# in the file
optirun bash --rcfile $tmp_file

My polished solution

Note that I use the fish shell, but the syntax should be simple enough to convert to bash if you feel inspired. Notable features include storing the log file for later reading if necessary, automatically figuring out if we should open a specific project file or the project window and fixing the project file path if it’s ambiguous.

function goUnreal
  set -l unreal_path $HOME/src/UnrealEngine/Engine/Binaries/Linux
  set -l tmp_file (mktemp)

  set -q UNREAL_LOG_FILE; and rm -f $UNREAL_LOG_FILE;
  set -U UNREAL_LOG_FILE (mktemp)

  echo "source ~/.bashrc" > $tmp_file
  echo "PS1=\"[Unreal Shell] ~> \"" > $tmp_file

  if count $argv[1] > /dev/null
    # Test if project file exists
    set -l project_file $argv[1]
    if test -e $project_file
      # Test if full path exists
      set project_file (pwd)/$project_file
      if test ! -e $project_file
        echo "Project file $project_file doesn't exist"
        return 1
      end
    end

    # Open specific uproject
    echo "$unreal_path/UE4Editor >$UNREAL_LOG_FILE \"$project_file\" 2>@1" >> $tmp_file
  else
    # Open project window
    echo "$unreal_path/UE4Editor >$UNREAL_LOG_FILE 2>@1" >> $tmp_file

    # Figure out the PID of the new editor instance that the project window spawned

    # Tokenize the string
    echo "editor_process=\$(ps -ejH | tail -n1)" >> $tmp_file
    # We only want the first token from the string so we just grab the first item here
    echo "editor_pid=( \$editor_process )" >> $tmp_file
    
    # Wait until the editor closes
    echo "while kill -0 \$editor_pid >/dev/null 2>@1; do sleep 1; done" >> $tmp_file
  end

  # Clean up and exit
  echo "rm -f $tmp_file" >> $tmp_file
  echo "exit" >> $tmp_file

  optirun bash --rcfile $tmp_file
end