Coding Range

Retrospective: Source 2 Leaks versus Reality

August 10th, 2014

Last week the Dota 2 Workshop Tools (Alpha) were released, which include a build of Dota 2 running on the Source 2 engine. I thought it would be interesting to see how it correlates with Source 2’s history of leaks, so first up is a bit of history.

History

19th July, 2010

On the 19th of July 2010, the Alien Swarm SDK was released. This was the only Source SDK code release to be based off the Left 4 Dead-series of branches, with the Source engine being somewhere between that of Left 4 Dead and that of Portal 2.

In the code was the first reference to Source 2. Tier0, a Valve library that gets embedded into just about every library and executable that Valve ship, contained hints of a new logging system in Source 2.

//-----------------------------------------------------------------------------
// ** NOTE FOR INTEGRATION **
// This was copied over from source 2 rather than integrated because 
// source 2 has more significantly refactored tier0 logging.
//
// A logging listener with Win32 console API color support which which prints 
// to stdout and the debug channel.
//-----------------------------------------------------------------------------
#ifndef _X360
class CColorizedLoggingListener : public CSimpleLoggingListener
{

The code shows support for printing console messages with arbitrary colours, not just the red, yellow and white of Source’s error, warning and info messages respectively.

6th August, 2012

Just over two years later, Source Filmmaker was released to the public. Source Filmmaker was originally a part of Team Fortress 2’s public beta, but it was removed from the final game with the promise of a return after the release of all of the Meet The Team videos. Meet The Pyro was released on the 27th of June, and shortly after came the toolkit used to make videos inside the Source Engine.

When ValveTime and Facepunch started pulling it to pieces, it became clear quickly that Source Filmmaker was laden with reference to Source 2, including:

  • Icons for ‘Source 2 Tools’, which may be built entirely using Qt and Python
  • 64-bit support
  • ‘gameinfo.gi’ replacing ‘gameinfo.txt’
  • New file formats - vmod, vproj, vmdl, vgame

3rd November, 2012

A few months later, 4chan’s /v/ visited Valve HQ for Gabe Newell’s 50th birthday, Gabe confirmed Source 2’s existence.

“We’ve been working on Valve’s new engine stuff for a while,” Newell responded, “we’re probably just waiting for a game to roll it out with.” When asked for confirmation that Source 2 would actually be a new engine and not an extension to previous iterations, Newell simply said “Yeah!” - The Escapist

1st June, 2013

In mid-2013, Steamworks became semi-public. Anyone was now able to sign up to Steamworks, view the documentation and download the SDK, after accepting the Steamworks NDA.

Some enterprising individuals soon discovered that the Steamworks partner website was not completely locked down, and was able to obtain the names associated with different Steam application IDs. Facepunch user DevinWatson discovered that application 235480 had the name ‘Left 4 Dead 3’.

Furthermore, an unnamed Valve Employee listed Source2 work on their profile, involving:

  • A “new component architecture system for game entities”
  • Integrating “the gameplay portion of… Left 4 Dead 2 into Source 2”

19th June, 2013

Just a few weeks later, Valve’s Jira bug tracker was accidentally made public. Of huge interest was the names of the various groups, which would reveal some sort of internal structure at Valve and thus the projects that Valve was working on. Whilst some of the groups were of little to no interest - as it appears that Jira was just synchronized with Active Directory - many were dead giveaways of either team structure or internal (Exchange) mailing lists, including:

  • Half-Life 3
  • L4D3
  • L4D3 Audio
  • L4D3 Developers
  • Source 2 Code Analysis Results
  • Source 2 Gameplay
  • Source2
  • Source2 Artists
  • Source2 Builds
  • Source2 Characters
  • Source2 Entities
  • Source2 Minidumps
  • Source2 Proto-games
  • Source2 Tools
  • Source2 Triage
  • Source2_Assertions
  • Source2_ContentAssertions
  • Source2Dev
  • Episode 3
  • Episode 3 Movie
  • src2sdk_assertions
  • src2sdk_contentassertions
  • src2sdk_minidumps
  • left4dead3_assertions
  • left4dead3_contentassertions
  • left4dead3_minidumps
  • Dota_Assertions
  • Dota_Contentassertions
  • Dota Minidumps

5th August, 2013

In the leadup to The International 3, some Valve fans were given a tour of Valve’s headquarters in Bellevue. One of the photos taken was of something not intended for fan viewing - Valve’s internal Perforce changelog.

Perforce Monitor

The web-based interface, seemingly built by Daniel Jennings, revealed that Valve were actively working on:

  • Source 2
    • VScript
    • Tier4 library (Source 1 only had Tier0 to Tier3)
    • Left 4 Dead 3
      • test_networking unit test
      • devtest level
    • ‘vagrp’ files in model_editor
  • SteamOps FBS
  • Other not-so-interesting things

Cross-referencing the names from the changelog to the Jira groups showed that the developers committing Source 2 changes were in the Source 2 and Half-Life 3 Jira groups:

  • Ted Carson
  • Kerry Davis
  • Ken Birdwell
  • Jay Stelly (Source 2 only, not Half-Life 3)
  • Jeff Hameluck

14th January, 2014

Sam Latinga a.k.a slouken was hired to join the Linux cabal in mid-2012. He created Simple DirectMedia Layer, most commonly known as SDL. On January 14th, he commited a changeset to SDL with the following included in the comment:

The reasoning behind this change is that source2 in -tools mode has a single OpenGL context that is used with multiple different windows. Some of those windows are created outside the engine (i.e. with Qt) and therefore we need to use SDL_CreateWindowFrom() to get an SDL_Window for those.

The Source Engine already used SDL for Linux and Mac OS X, but this changeset suggested that Source would use SDL and Qt for the Source 2 Tools, and possibly for the engine itself.

27th January, 2014

On January 27th, there were not one but two big leaks.

Firstly, NeoGAF user crazy buttocks on a train, a.k.a CBOAT posted screenshots from a confidential presentation which clearly showed Left 4 Dead 2 being rebuilt in ‘Source 2.0’ with hugely improved detail on the Plantation level.

Source2 Presentation - Plantation Level in L4D2

Shortly afterwards, Facepunch user testinglol posted a screenshot from some sort of Perforce web interface.

P4 Web - //source2/main/game

It shows the source tree of //source2/main/game, including most notably:

  • cs2 (Counter-Strike 2)
  • dota (Dota 2)
  • dota_core
  • dota_imported
  • hl3 (Half-Life 3)
  • hl3_imported
  • left4dead2_imported
  • left4dead2_source2
  • left4dead3
  • tf (Team Fortress 2)
  • tf_imported
  • sdktools
  • An installscript for Steam Application ID 244670* l>

4th March, 2014

On March 4th, Gabe Newell along with a couple of other developers did an Ask Me Anything session on Reddit. When asked about Source 2, this was their response:

The biggest improvements will be in increasing productivity of content creation. That focus is driven by the importance we see UGC having going forward. A professional developer at Valve will put up with a lot of pain that won’t work if users themselves have to create content.

(Note: UGC is User-Generated Content)

6th August, 2014

On August 6th, the Dota 2 Workshop Tools (Alpha) were released to the public, which included a 64-bit Windows build of Dota 2 running on top of Source 2.

Expectations vs Reality

“significantly refactored tier0 logging”

This can be seen in VConsole2 and Dota 2. Logging now logs to different channels, suchas ‘InputBindSystem’ and ‘SoundSystem’. Logs can be filtered, searched through, and individual channels can be broken out into new VConsole tabs.

Different channels can be bound to different colours, as well as having different colours for Default, Alternate, DETAILED, MESSAGE, WARNING, ASSERT and ERROR.

Source2 - VConsole2

Source2 - VConsole2 Channel Settings

Expectation: Fulfilled.

Source 2 Tools, using SDL, Qt and Python

I haven’t done a detailed analysis of the tools, but dota_ugc/game/bin/win64 includes pyside-python2.7.dll, python27.dll, pythoncom27.dll, pythoncomloader27.dll pywintypes27.dll, QtCore4.dll, QtGui4.dll, QtOpenGL4.dll, SDL2.sll, shiboken-python2.7.dll.

Michael Sartain, one of Valve’s Linux developers, has also blogged about Qt on a couple occasions.

Source Filmmaker is also built on Qt, so given the above it makes sense that the tools are built on Qt.

Update: It’s definitely using those libraries:

Source2 - VConsole2 Channel Settings

Expectation: Fulfilled.

64-bit Support

Source 2 doesn’t just support 64-bit, but the Dota 2 Workshop Tools are 64-bit only. Valve have said 32-bit is coming soon.

Expectation: Fulfilled.

gameinfo.gi file to replace gameinfo.txt

The Source 2 build of Dota 2 uses gameinfo.gi, which is a Valve KeyValues format file that looks like an extention of Source’s gameinfo.txt.

Expectation: Fulfilled.

New file formats - vmod, vproj, vmdl, vgame

The Dota 2 Workshop Tools use vmat, vtex, vfont, vmap, vpcf, vrman, vrmap, vsndevts, vsndstck, vsurf and vmdl files.

Expectation: Fulfilled.

More than just an extention to previous iterations of Source

Just about everything is different. Not going to provide a solid in-depth analysis here, but I feel confident enough to say

Expectation: Fulfilled.

“new component architecture system for game entities”

I haven’t seen any of this yet, but I don’t know enough about the entity system in Source to confidently comment on this.

Expectation: Unknown.

Left 4 Dead 2 on Source 2

Nothing yet.

Expectation: Still Waiting.

Half-Life 3 on Source 2

Nothing yet.

Expectation: Still Waiting.

Left 4 Dead 3 on Source 2

Nothing yet.

Expectation: Still Waiting.

Source 2 SDK

The only SDK we’ve seen so far is the Workshop Tools.

Expectation: Partially Fulfilled.

Dota 2 on Source 2

Apart from the dota, dota_core and dota_imported references, the community missed something that’s incredibly obvious in retrospect.

Have a look through the Jira groups / Active Directory groups / Exchange lists again. There’s a pattern:

  • Source2 Minidumps
  • Source2_Assertions
  • Source2_ContentAssertions
  • src2sdk_assertions
  • src2sdk_contentassertions
  • src2sdk_minidumps
  • left4dead3_assertions
  • left4dead3_contentassertions
  • left4dead3_minidumps
  • Dota_Assertions
  • Dota_Contentassertions
  • Dota Minidumps

Dota has three groups that fall in line with the other confirmed Source 2 group sets (Source2, src2sdk, left4dead3). These are the only sets of groups that have Assertions, Contentassertions and Minidumps groups.

Expectation: Fulfilled (Alpha, the rest is probably coming soon).

Source 2 to have a Tier 4 library

engine2.dll, client.dll and server.dll contain references to CTier4AppSystem. Below are the strings from engine2.dll:

.?AV?$CTier4AppSystem@VIEngineSound@@$0A@@@
.?AV?$CTier4AppSystem@VIVEngineClient2@@$0A@@@
.?AV?$CTier4AppSystem@VIVEngineServer2@@$0A@@@
.?AV?$CTier4AppSystem@VIUploadGameStats@@$0A@@@
.?AV?$CTier4AppSystem@VINetworkStringTableContainer@@$0A@@@
.?AV?$CTier4AppSystem@VIBenchmarkService@@$0A@@@
.?AV?$CTier4AppSystem@VIEngineService@@$0A@@@
.?AV?$CTier4AppSystem@VIGameResourceService@@$0A@@@
.?AV?$CTier4AppSystem@VIGameUIService@@$0A@@@
.?AV?$CTier4AppSystem@VIInputService@@$0A@@@
.?AV?$CTier4AppSystem@VIMapListService@@$0A@@@
.?AV?$CTier4AppSystem@VINetworkClientService@@$0A@@@
.?AV?$CTier4AppSystem@VINetworkServerService@@$0A@@@
.?AV?$CTier4AppSystem@VINetworkService@@$0A@@@
.?AV?$CTier4AppSystem@VIRenderService@@$0A@@@
.?AV?$CTier4AppSystem@VISoundService@@$0A@@@
.?AV?$CTier4AppSystem@VISplitScreenService@@$0A@@@
.?AV?$CTier4AppSystem@VIStatsService@@$0A@@@
.?AV?$CTier4AppSystem@VIToolService@@$0A@@@
.?AV?$CTier4AppSystem@VIUserInfoChangeService@@$0A@@@
.?AV?$CTier4AppSystem@VIVDebugService@@$0A@@@
.?AV?$CTier4AppSystem@VIEngineServiceMgr@@$0A@@@
.?AV?$CTier4AppSystem@VIHostStateMgr@@$0A@@@
.?AV?$CTier4AppSystem@VIKeyValueCache@@$0A@@@
.?AV?$CTier4AppSystem@VITextMessageMgr@@$0A@@@
.?AV?$CTier4AppSystem@VIGameEventSystem@@$0A@@@

Expectation: Fulfilled.

Team Fortress 2 on Source 2

Nothing yet.

Expectation: Still Waiting.

Steam Application ID 244670

There’s a file in dota_ugc/game/dota named steam_244670.inf:

Source2 - VConsole2 Channel Settings

The contents of the file follows the standard steam.inf convention:

ClientVersion=23
ServerVersion=23
PatchVersion=1
ProductName=dota2_s2_main
appID=244670

The product name above is dota2_s2_main, which likely means Dota 2 Source 2 Main.

Expectatation: Fulfilled.

Focus on User-Generated Content

If you really need to read this section, just close this article and go play around with the Dota 2 Workshop Tools. Seriously. The Source 2 tools that ship with the Dota 2 Workshop Tools blow the Source 1 tools out of the water, and while I haven’t tried to build anything serious myself, ‘everyone’ says it’s incredibly easy to create content with.

Expectation: Fulfilled.