|
Introduction
Welcome to the "Gotcha" section of TGEA's 1.8 porting guide. If you
are unfamiliar with the term "Gotcha" (quotes and all), imagine writing
valid code in the engine that results in a bug or crash. The code
worked in a previous version of TGEA, but is busted in 1.8. What you
are experiencing is an engine modification you are not aware of messing
up your work.
This is the engine's way of saying "Gotcha!" We've taken some time to
document common issues you will run into while porting your previous
work into TGEA 1.8, or writing your own code that is dated when
compared to new GFX2 code. The most critical issues are listed in this
article, but more may be added at a later date. If you run into a
graphical issue that does not result in a crash or error, be sure to
check this page again to see if you've fallen into a "Gotcha"
situation.
Main Loop Change
This "Gotcha" is one of the first modifications you need to make if you are merging an older project. Each game project has a main.cpp file. This source file is local to your project, not the engine's source. For example, the T3D main.cpp can be found in the GameExamples/T3D/source/ directory. Within this file is the entry point for your game:
// Entry point for your game.
//
// This is build by default using the "StandardMainLoop" toolkit. Feel free
// to bring code over directly as you need to modify or extend things. You
// will need to merge against future changes to the SML code if you do this.
S32 TorqueMain(S32 argc, const char **argv)
{
...
}
The actual main loop used to be a standard function call:
// Main loop
StandardMainLoop::doMainLoop();
However, if you try running this on a 1.8 build, the application
will run once and exit immediately. To rectify this, you must use a
standard C while loop to keep your application running:
// Main loop
while(StandardMainLoop::doMainLoop());
Setting Shader Constant Buffers
Before you perform a draw call, you must remember to set your shader
constant buffers (GFXShaderConstBufferRef). If you forget to do this,
you can end up with odd graphical glitches that do not actually produce
errors/crashes/warnings.
Pseudo-Code
// Get projection matrix first
proj = GFX->getProjectionMatrix();
// Here, we are asssigning our shader constant buffer properties
myShaderConstBuffer->set(mModelViewProjSC, &proj);
myShaderConstBuffer->set(mOpacityMapSC, (S32)0);
mShaderConsts->set(mLightMapSC, (S32)1);
// *** SET SHADER CONSTANT BUFFER ***
GFX->setShaderConstBuffer(mShaderConsts);
// Now you can safely use draw calls
...
Actual Code: Precipitation
In Precipication.h, you will find various shader constant buffer variables. Here are a few examples:
GFXShaderConstBufferRef mSplashShaderConsts;
GFXShaderConstHandle* mSplashShaderModelViewSC;
GFXShaderConstHandle* mSplashShaderFadeStartEndSC;
The Render Delegate for Precipication is called renderObject(...).
Within this function, you can see how we have modified the file to
avoid this "Gotcha". First, we actually initialize the shader constant
buffer:
if (mSplashShaderConsts.isNull())
{
mSplashShaderConsts = mSplashShader->mShader->allocConstBuffer();
if (mSplashShaderConsts)
{
mSplashShaderModelViewSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$modelView");
mSplashShaderFadeStartEndSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$fadeStartEnd");
mSplashShaderCameraPosSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$cameraPos");
mSplashShaderAmbientSC = mSplashShaderConsts->getShader()->getShaderConstHandle("$ambient");
}
}
In the above code, we check to see if our shader constant buffer
is nullified. If it is, we allocate a buffer. If that succeeded, we
initialize shader constant handles. Once everything is initialized, we
address the "Gotcha" in two spots. One is right below the
initialization:
if (mSplashShaderModelViewSC)
mSplashShaderConsts->set(mSplashShaderModelViewSC, proj);
if (mSplashShaderFadeStartEndSC)
mSplashShaderConsts->set(mSplashShaderFadeStartEndSC, fadeStartEnd);
The other happens further down, just before rendering occurs:
if (mSplashShader)
{
if (mSplashShaderConsts.isValid())
GFX->setShaderConstBuffer(mSplashShaderConsts);
mSplashShader->getShader()->process();
}
As you can see, the main point of this section is addressed. If we
are using the Splash Shader, we check to see if we have a valid shader
constant buffer for splash rendering. If so, GFX sets the shader
constant buffer before we render.
Transpose Gotcha
Do not transpose transform matrices before passing a shader. Doing
so will result in odd visuals, since you will be passing in the wrong
matrix. You might see objects flickering and shooting off into infinity
from your perspective. We've avoided doing this in the newer GFX2 code,
but if you need to see an example of where this "Gotcha" was corrected,
you can look at the fxFoliageReplicator class. Open
fxFoliageReplicator.cpp.
The class's Render Delegate is renderObject(...).
If you jump down to where the shader constant handles are set in this
function (around line 1632), you will see the "Gotcha" being handled:
// Set up our shader constants
// Projection matrix
MatrixF proj = GFX->getProjectionMatrix();
//proj.transpose();
if (mFoliageShaderProjectionSC)
mFoliageShaderConsts->set(mFoliageShaderProjectionSC, proj);
In the above code, we get the projection matrix from GFX. A few
lines down, we set the shader. As you can see, we commented the
transpose function. Keep this in mind when you are writing similar
rendering code.
Name Consistency
This can be a very frustrating "Gotcha", as the likelihood of
repeating the mistake is high. You must avoid inconsistent naming of
shader variables. As an example, we are going to examine a water shader
which has been updated to work with TGEA 1.8.
You will want to open GameExamples/Stronghold/game/shaders/water/waterCubeReflectRefractV.hlsl with a text editor of your choice. Pay close attention to what
variables are assigned to which registers. The previous version of this
shader used a different name for its light vector:
uniform float3 lightVec : register(C21),
In past versions, you would directly set the register and what
you named lightVec did not matter. You were mainly concerned with using
the appropriate variables defined in shaderConsts.h.
Now, using "lightVec" as the name will result in odd water rendering,
such as completely black. Compare this to TGEA 1.8's version of this
shader:
uniform float3 inLightVec : register(C21),
lightVec has been renamed to inLightVec. Using the new
variable name solves the bad water rendering. This "Gotcha" is easy to
fix, so long as you remember the following information. If you wish to
use TGEA's material system with a shader, the variables you assign to a
register must match the ones used in the shader gen related source
files. If you have not already tracked these files down, let's take a
look.
Navigate to the engine/source/shaderGen directory. Open shaderGenVars.h and shaderGenVars.cpp. In the header, a struct called ShaderGenVars is defined. Pre-TGEA 1.8 used VC_LIGHT_DIR1, so scroll down until you see something similar. You should come across this:
const static String light1Direction;
How does this relate to the naming consistency issue? Switch to shaderGenVars.cpp and browse for light1Direction again.
const String ShaderGenVars::light1Direction("$inLightVec");
Jackpot! This is where inLightVec is assigned, and this is how the
shaders tie in with TGEA's custom material system. So, to reiterate,
make sure the shader variable names you use in a shader file match the
ones declared in the engine. Don't forget the $ symbol, either!
Shader Compile Failure
This is a simple "Gotcha" to explain, but you need to keep an eye
out for it. If your shader is not being displayed, or has bad results,
check for a shader compile failure listed in the console. The engine
may continue to run with only a warning about the compile failure. If
this happens, go to the shader and check for any syntax errors or
improper logic.
Set StateBlock
The introduction of StateBlocks introduces a new "Gotcha", but only
if you are coding in the engine. When you declare a GFXStateBlocKData
in script, the StateBlock is automatically set. This isn't the case
when you are working in the engine. If you need an example, let's take
a look at the Sky class. Open Sky.cpp, then scroll down to renderSkyBox(...).
GFX->setStateBlock(mSkyBoxSB);
The very first operation after the variables are declared is setting the StateBlock. This must happen before any rendering calls are made. Another issue with setting
a StateBlock is the overriding that can occur. If you have declared a
GFXStateBlock in script, it will override any C++ counterpart. For
instance, if you are making changes to the terrain StateBlocks in C++,
you will most likely not see any results since the TorqueScript version
has taken over.
RenderDelegate
When reading about a Render Delegate,
this "Gotcha" is stated but not in a way you might have noticed. Prior
to TGEA 1.8, objects were automatically rendered using the same
function signal. This function was renderObject(), which every object
had to use if it wanted to be drawn on screen.
With the introduction of Render Delegate,
this is no longer the case. This is actually a huge benefit, as you can
now name an object's render function more appropriately. For instance,
if you could change the Sky class's rendering function name to
"renderSky", so long as you modify its Render Delegate at the time.
This brings up another "Gotcha" related to RenderDelegate. If you keep
the function name "renderObject", you must change the function
parameters and how it is set up. Again, let's use Sky as an example.
Before 1.8, the prepRenderImage(...) class method contained this
codeblock:
if (state->isObjectRendered(this))
{
RenderInst *ri = gRenderInstManager.allocInst();
ri->obj = this;
ri->state = state;
ri->type = RenderInstManager::RIT_Sky;
gRenderInstManager.addInst( ri );
}
Compare that to how TGEA 1.8 sets up the RenderInst pointer:
if (state->isObjectRendered(this))
{
ObjectRenderInst *ri = gRenderInstManager->allocInst<ObjectRenderInst>();
ri->mRenderDelegate = mRenderDelegate;
ri->state = state;
ri->type = RenderPassManager::RIT_Sky;
gRenderInstManager->addInst( ri );
}
There are several changes to this code block, but we will
address all of them in the Sky Class Comparison document. For now,
let's focus on the assignment of mRenderDelegate.
Sky's RenderDelegate is assigned to Sky::renderObject(...). This
function existed before TGEA 1.8 and continues to exist in the current
version of TGEA. However, there is not a 1 to 1 compatibility between
the versions. The parameters that are passed have changed:
Previous TGEA Version
void renderObject(SceneState *state, RenderInst *ri);
TGEA 1.8 Version
void renderObject(ObjectRenderInst *ri, BaseMatInstance* );
Instead of taking in SceneState and RenderInst pointers, the new renderObject function takes in an [[1]]
and a BaseMatInstance*. The main point is previous class
implementations of renderObject(...) are no longer compatible with TGEA
1.8 renderObject(...).
Remember this point or you could end up with a compiler error that
makes no sense and is hard to track down. For example, Sky's
RenderDelegate member variable is bound in the constructor:
mRenderDelegate.bind(this, &Sky::renderObject);
If you do not modify the old Sky::renderObject(...) parameters, you will receive this error:
error C2660: 'fastdelegate::FastDelegate2<Param1,Param2,RetType>::bind' : function does not take 2 arguments
Notice the lack of information pointing you to the renderObject(...) function.
GLSL Texture Order
By supporting OpenGL, we have to adhere to certain rules we were not
obligated to follow in past versions of TGEA. One such rule is enforced
when creating ShaderData objects in TorqueScript. ShaderData must now
contain DX and OGL shader file references, as well as a new element.
If you open C:/torque/TGEA_1_8_Repo/GameExamples/Stronghold/game/common/clientScripts/shaders.cs, you will see a perfect example of this "Gotcha":
new ShaderData( _DebugInterior_ )
{
DXVertexShaderFile = "shaders/debugInteriorsV.hlsl";
DXPixelShaderFile = "shaders/debugInteriorsP.hlsl";
OGLVertexShaderFile = "shaders/gl/debugInteriorsV.glsl";
OGLPixelShaderFile = "shaders/gl/debugInteriorsP.glsl";
OGLSamplerNames[0] = "$diffuseMap";
pixVersion = 1.1;
};
Notice how vertex and pixel shader files are represented for both DX and OGL. That's not all. Look at the OGLSamplerNames[0] variable and the $diffuseMap value assigned to it. This is a new section of ShaderData, which is
necessary because a GL shader does not know anything about texture
order. Unlike a hlsl shader, it does not assume anything.
Let's compare the pixel shaders listed in the above ShaderData object. Specifically, let's focus on the diffuseMap usage:
DX (debugInteriorsP.hlsl)
//-----------------------------------------------------------------------------
// Structures
//-----------------------------------------------------------------------------
struct ConnectData
{
float2 texCoord : TEXCOORD0;
};
struct Fragout
{
float4 col : COLOR0;
};
//-----------------------------------------------------------------------------
// Main
//-----------------------------------------------------------------------------
Fragout main( ConnectData IN,
uniform sampler2D diffuseMap : register(S0),
uniform float4 shadeColor : register(C0)
)
{
Fragout OUT;
OUT.col = shadeColor;
OUT.col *= tex2D(diffuseMap, IN.texCoord);
return OUT;
}
OGL (debugInteriorsP.glsl)
uniform sampler2D diffuseMap;
uniform vec4 shadeColor;
varying vec2 TEX0;
void main()
{
vec4 diffuseColor = texture2D( diffuseMap, TEX0 );
gl_FragColor = diffuseColor;
gl_FragColor *= shadeColor;
}
As you can see in the OGL shader, the texture order is explicitly expressed in the first line of the main() function. Since you must have a OpenGL shader equivalent to the DX version, keep this texture order "Gotcha" in mind.
Shared Shader Global Space
This is a simple "Gotcha" to remember. Pixel and vertex shaders
constants share the same global space. You must name your variables
differently. Let's say you are using $modelView for a pixel shader
const and vertex shader const. The system will read in and use the
vertex shader first.
Now, if a variable contains the same data for both shader consts, you
should not have a problem. However, this is not always going to be the
case and you should keep this in mind.
Resource Loading Change
The new Resource Manager introduces a "Gotcha", which directly
affects the loading of texture files, such as bitmaps and DDS.
Previously, the GBitmap and DDSFile classes would return a class
pointer. For example:
GBitmap* bitmap = GBitmap::load( path );
This is no longer the case. The new Resource Manger actually
returns a resource handle. The GFXTextureManager class has a perfect
example. An extremely important class function GFXTextureManager::createTexture(...). Within this function, we can see the Resource Manager in action:
// MP: Load a DDS file
Resource<DDSFile> dds = DDSFile::load( path );
// MP: Load a bitmap file
Resource<GBitmap> bitmap = GBitmap::load( path );
As you can see, instead of returning a class pointer (GBitmap* or
DDSFile*), the ::load function now returns a handle to the
resource, via a templated class that acts as a wrapper
(Resource<class T>).
What is a CSF?
Simply put, CSF stands for Compiled Shader File.
CSF files are automatically created, depending on a few conditions
checked by the engine internally. CSF files have a similar
functionality as DSO files.
When you compile a TorqueScript file to DSO, the game still
reads it properly, but the script itself is converted to binary form
making it more secure. When your shader file is compiled to CSF, it can
no longer be modified by an end user.
When a shader declaration is read, the engine initializes it internally:
bool GFXD3D9Shader::init( const Torque::Path &vertFile,
const Torque::Path &pixFile,
F32 pixVersion,
const Vector<GFXShaderMacro> ¯os )
{
...
}
Inside the the init function, there is a check to see if the system needs to compile the shader:
if ((GFXD3DX.isLoaded) && (!Con::getBoolVariable("$shaders::forceLoadCSF", false)))
{
_compileShader( vertFile, vertTarget, d3dXMacros, mVertexConstBufferLayoutF, mVertexConstBufferLayoutI);
...
In the engine's source, there is a preprocessor definition named TORQUE_ENABLE_CSF_GENERATION. If this has been defined, the shader file will be compiled and saved out as a CSF:
#ifdef TORQUE_ENABLE_CSF_GENERATION
// Ok, we've got a valid shader and constants, let's write them all out.
if (!_saveCompiledOutput(filePath, code, bufferLayoutF, bufferLayoutI))
Con::errorf("Unable to save shader compile output for: %s", filePath);
#endif
Material Initialization
The way materials are initialized has changed. We can look at the MatInstance class (engine/source/materials/matInstance.h) to see this modification:
Old Initialization Function
void init( SceneGraphData &dat, GFXVertexFlags vertFlags );
New Initialization Function
virtual void init(const GFXMaterialFeatureData::FeatureListHandle instanceData, const GFXMaterialFeatureData::FeatureListHandle globalData, GFXVertexFlags vertFlags);
Previously, the init(...) function took in a SceneGraphData
reference which was parsed for information and features. In TGEA 1.8,
we pass in two very different parameters in addition to the original vertFlags parameter:
- GFXMaterialFeatureData::FeatureListHandle instanceData -
Should contain all of the features that this instance can support. For
example, if it supports lightmaps then LightMap should be turned on,
but if it's an instance for a mesh without lightmaps, then lightmap
should be off.
- GFXMaterialFeatureData::FeatureListHandle globalData -
Used to turn features on and off at a more broad level. By default,
just get the one from the MaterialManager. This and instance data are
AND'ed together and used with the material setup to figure out which
features to turn on and off.
- GFXVertexFlags vertFlags - The vertex type to be used with this instance.
OGLSamplerNames Field Name Change
Prior to TGEA 1.8, the exposed form of the ShaderData class
(ConsoleObject used in TorqueScript) used a field named
"OGLSamplerNames". The internal variable used by this field was
ShaderData::mOGLSamplerNames, found in
engine/source/materials/shaderData.h.
void ShaderData::initPersistFields()
{
// ... previous and remaining code not shown
String mOGLSamplerNames[TEXTURE_STAGE_COUNT];
}
This variable was used in many places throughout the engine, but was
exposed to TorqueScript in ShaderData:initPersistFields() (found in
engine/source/shaderData.cpp):
addField("OGLSamplerNames", TypeRealString, Offset(mOGLSamplerNames, ShaderData), TEXTURE_STAGE_COUNT);
The latest version of TGEA has renamed both the class member
variable and the script implementation. The following code represents
the name change:
Declaration of class variable in engine/source/shaderData.h:
String mSamplerNames[TEXTURE_STAGE_COUNT];
Exposing variable to script in engine/source/shaderData.cpp:
void ShaderData::initPersistFields()
{
// ... previous and remaining code not shown
String mSamplerNames[TEXTURE_STAGE_COUNT];
}
Automated Mapping of samplerNames
A new class method has been added to ShaderData, which can be found in enginer/source/materials/shaderData.h/.cpp:
void mapSamplerNames(GFXShaderConstBufferRef constBuffer);
The new function definition can be found below:
//--------------------------------------------------------------------------
// mapSamplerNames
//--------------------------------------------------------------------------
void ShaderData::mapSamplerNames(GFXShaderConstBufferRef constBuffer)
{
if (constBuffer.isNull())
return;
// Map the stage numbers to the constants
for (U32 i = 0; i < TEXTURE_STAGE_COUNT; i++)
{
if (mTexHandlesSC[i])
constBuffer->set(mTexHandlesSC[i], (S32)i);
}
}
The purpose of this new method is to automate the mapping of the
"samplerNames" fields from script to the actual shader buffer
constants. You should use this function anytime after you use
GFXShader::allocConstBuffer() and pass it in your newly created
GFXShaderConstBufferRef so it can map the actual texture stage values
into your constant buffer's parameters.
You will need to do this to allow GLSL shaders to properly sample from
the right texture register. A simple example can be found in
GroundCover class, specifically GroundCover::_initShader(). Open up engine/source/T3D/fx/groundCover.cpp.
Scroll down to the function I just mentioned. Halfway through the
function, after the samplers are set up, you will see the example:
void GroundCover::_initShader()
{
// ... previous and remaining code not shown
mConstBuffer = mBBShader->mShader->allocConstBuffer();
if (mConstBuffer)
mBBShader->mapSamplerNames(mConstBuffer);
}
</pre>
As you can see, once the allocConstBuffer is called successfully we call mapSamplerNames with thew newly defined mConstBuffer.
November 2008 DirectX SDK Shader Bug
There was a bug in the shader compiler in the November 2008 DirectX SDK (as referenced in this GarageGames forum thread: TGEA 1.7.1 and DirectX Nov. 2008 - broken ). A workaround was added for it that is specific to the Nov 2008 DXSDK (also detailed in that thread).
However, we recommend you stick with the August 2008 DXSDK
(also linked in that thread). Our contacts at Microsoft informed us the
bug report was recorded and should be fixed in the March 2009 DirectX
SDK.
|