Turtle Artefacts

Turtle is a lighting and baking renderer originally developed by Illuminate Labs, available as a beta in June 2003 and released as 1.0 version the 31 June 2004. Predominantly used for fast baking textures in the game industry in conjunction with Beast, it is owned by Autodesk since July 2010 when they acquired the company Illuminate Labs.

It was first bundled with the 2011 Autodesk Maya Entertainment Creation Suite Premium, and then for everyone in Maya 2014 even though it seems to fall into abeyance… strange thing coming from Autodesk.

It can be very useful for ambient occlusion, it is much more faster and cleaner than mental ray… and clunky, especially when using the MEL commands. The documentation is incomplete and nearly non-existent.

One problem I stumbled across was those crazy artefacts you get sometimes when baking Ambient Occlusion. From what I have seen, those come when you try to bake several meshes with the same shaders or with separate UV shells AND if you try to bake in more than 2K.

Taken from this sadly very alone post. Others can be found here and here (which the user have found the solution, but does not say it, great) and here.

The problem comes from the Bilinear Filter and this bug seems to be there since at least 5-6 years. The weird thing is that if you deactivate the bilinear filter in the Texture Bake Settings it works but not if you launch the bake through the command line. It’s like the bilinear filter setting is never taken into account no matter what. And nothing can be found in the doc.

I then activated the Echo All Command in the script editor and tried a GUI render. We can see it is doing those things:

renderWindowRender redoPreviousRender renderView;
Interactive render command : ilrBakeCallback(2000, 1000 1, 1, "persp", " -layer defaultRenderLayer") // Notice how the "-layer" flag is passed, optional arguments yay!
ilrTextureBakeCmd -target "blabla_mshShape" -target "blabla2_mshShape" -frontRange 100 -backRange 100 -frontBias 0 -backBias 0 -transferSpace 1 -selectionMode 0 -mismatchMode 0 -envelopeMode 0 -ignoreInconsistentNormals 1 -considerTransparency 0 -transparencyThreshold 0.001000000047 -camera "persp" -normalDirection 0 -shadows 1 -alpha 1 -viewDependent 0 -orthoRefl 1 -backgroundColor 0 0 0 -frame 1 -bakeLayer TurtleDefaultBakeLayer -width 4096 -height 4096 -saveToRenderView 1 -saveToFile 0 -uvRange 0 -uMin 0 -uMax 1 -vMin 0 -vMax 1 -uvSet "" -tangentUvSet "" -edgeDilation 0 -bilinearFilter 1 -merge 1 -conservative 0 -windingOrder 1 -fullShading 1 -useRenderView 1 -layer defaultRenderLayer

We could just use the ilrBakeCallback function, but instead we’ll use the whatIs command to look what is inside ilrBakeCallback.

whatIs ilrBakeCallback;
// Result: Mel procedure found in: /Maya_directory/turtle/scripts/turtle/ilrBake.mel //

In that function, this part is the interesting one:

switch ($bakeTo)
{
    case 1: // Texture
        $cmd = ilrGetTextureBakeCmdString($layer, "");
        $cmd += " -useRenderView 0";
        $cmd += $extraOptions;
        break;
    case 2: // Vertices
        $cmd = ilrGetVertexBakeCmdString($layer, "");
        $cmd += $extraOptions;
        break;
}

You can see it calls ilrGetTextureBakeCmdString in case of a texture bake, and ilrGetVertexBakeCmdString for a vertex bake. When you look at ilrGetTextureBakeCmdString you immediately notice the call to ilrTextureBakeCmd, the command specified in the documentation, following by all the flags for that function… but there are more flags in there than in the documentation and more importantly the -bilinearFilter one!

If you add -bilinearFilter 0 to your ilrTextureBakeCmd command, the artefacts disappear! Here is a list of all the different flags, and here are all the 45 flags not listed in the documentation for the ilrTextureBakeCmd not listed in the documentation.

Enjoy!

-alphaBlend
-alphaMax
-alphaMin
-alphaScale
-bilinearFilter
-clamp
-colorSet
-considerTransparency
-directory
-dirOcc
-dirOccExponent
-dirOccSamples
-dirOccVector
-filter
-filterNormalDev
-filterShape
-filterSize
-lua
-luaFile
-maxSamples
-minSamples
-overwriteColorSet
-ptm
-rgbBlend
-rgbMax
-rgbMin
-rgbScale
-rnm
-rnmSamples
-samplingMode
-saveToColorSet
-skipSelect
-sph
-sphBands
-sphConeAngle
-sphOutput
-sphSamples
-sphSampleType
-sphSignLess
-sphSpaceType
-sss
-tangentUvSet
-transparencyThreshold
-useBlending
-vertexBias