In this post, I present a honeypot approach to combating group-think and quality deterioration in article selection on social news sites.

Article Honeypots

A honeypot is an article that is link-bait or otherwise in direct violation of the site's guidelines, but is intentionally submitted by an admin as a test to see if users inappropriately upvote it². For each user, three scores are tracked: the number of honeypots seen, the number of honeypots upvoted, and the number of honeypots flagged. A user's seen count increments when they load a page with a honeypot article displayed³. If a user upvotes one of the honeypots, their upvoted score is incremented; if the user flags the honeypot, their flagged score is increment. We then take the ratio of the difference of the flagged and upvoted scores, and the seen score to get a honeypot ratio, h:

where:

is the target user
is the number of honeypots the user flagged
is the number of honeypots the user upvoted
is the total number of honeypots seen by the user

This produces a ratio in the range [-1,1]. We may want to punish people who excessively flag everything, otherwise they'll always have a max h score⁴:

where:

is the total number articles a user has flagged

The range of this h-ratio is [-2,1). However, because is very likely going to be much larger than for every user, we would expect that no user will ever receive a score near 1 in practice. If the h-ratio of a user is less than an admin-specified threshold, we flag the user as detrimental to the overall quality of the site and their upvotes would either be discounted or ignored entirely.

Implicit Honeypots

Since it's not always feasible to expect admins to find or label honeypots, it would be nice to have some way to crowdsource honeypots implicitly. To do this, we take the top N users, ranked by their explicit honeypot ratio, and label them as "super flaggers". For each article, we track the percentage of super flags they've received and if this percentage is above an admin-specified threshold, we automatically label that article as a honeypot:

function getSuperFlaggers(honeypots, users, superFlaggersCount):
    # Order users by their honeypot ratio, in descending order
    ranked_users = users.sort(u => u.honeypotRatio)

    return ranked_users.subset(0, superFlaggersCount)

function isHoneypot(article, threshold, superFlaggers):
    count = 0
    seen = 0
    foreach sf in supperFlaggers:
        seen += 1 if seen(sf, article)
        count += 1 if flagged(sf, article)
    percent = count / seen
    return percent >= threshold

It would make sense to check this value every time an article hits a certain number of upvotes. One would also likely want to ensure that seen has reached a high enough value to be significant before checking. If an article is labeled as a honeypot, then all users who have already seen the article should have their values retroactively updated.

Conclusion

The above post presented two reasonable approaches to improving the quality of the front page on social news sites like HackerNews. Social news sites have long been believed to suffer from the deterioration over time, with recent evidence supporting that belief. While anecdotal evidence suggests hiding comment karma has helped improve discussion quality, article selection quality has remained largely unchanged⁵. The honeypots approach I described above may help ebb the flow of upvotes to link-bait and inappropriate articles by enabling the community to moderate itself implicitly.

If you liked this article, please vote it up on HackerNews. Assuming you think it meets the quality guidelines.

Footnotes

While all these attempts were good ideas, I think there is an even better metric that should be used here: the Sharpe ratio.

Risk adjusted returns

When you look to invest in an asset, it's often important to consider the risk of an investment as well as the overall historical returns. Assuming you have no external knowledge of the asset, looking at the volatility of its returns is a pretty good estimate of risk. Ideally, you'd like to see super high returns with no volatility– just a steady stream of money rolling in day after day.¹ The Sharpe ratio is a way to combine the returns of an asset and its historical risk into a single number.

Simply defined, the Sharpe ratio is the return of the asset minus the risk free rate, all divided by the standard deviation of returns. For instance, if we have an asset which has returned an average of 10% per year and an annualized volatility of 20%, and we assume that we could park our money in a savings account for a risk-free 1% per year, then the Sharpe ratio for this asset is (10 - 1) / 20 = 0.45.

Naive Sharpe karma

How does this translate to a social news site like HackerNews? First, let's consider each user to be an investable asset. Each time a comment or story submission is made, that user is generating a return.² Since each comment starts with a default score of 1, let's go ahead and make that the risk free rate of return. So the karma metric I'm proposing would be:

where:
is the user
is the number of comments for user
is the karma score for the ith comment of user
is the standard deviation of the karma scores for user

In programmer-speak, the pseudo-code for this is³:

function karmaSharpe(user) {
    if(user.comments.length == 0)
         return 0;

    var numerator = 0;
    foreach(var comment in user.comments)
         numerator += (comment.karma - 1) / user.comments.length;

    var denominator = stdev(x => user.comments.karma);
    if(denominator < 1)
        denominator = 1;

    return numerator / denominator;
}

A better metric

The Sharpe ratio is certainly an improvement over the previous two metrics. Totals give a very low-fidelity view of the user's contribution and averages make no distinction between consistent, high-quality members and more skewed users who have a couple really highly-upvoted submissions. The Sharpe ratio addresses both of these shortcomings by including an additional dimension in the calculation: volatility.

The metric is still not perfect, however. A good metric would include time in the formula, since a high-quality user would be consistently contributing every day. An improved metric is below:

where:
is the user
is the number of days user has been registered
is the number of comments for user made on day
is the karma score for the comment of the day for user
is the standard deviation of the karma scores for user

And in pseudo-code:

function timeAdjustedKarmaSharpe(user) {
    if(user.comments.length == 0)
         return 0;

    var excessReturns = dailyAverageExcessReturns(user);
    return karmaSharpe(excessReturns);
}

function dailyAverageExcessReturns(user) {
    var groups = user.comments.groupBy(c => c.Date);
    var excessReturns = groups.select(g => g.sum(c => c.karma - 1) / g.count);
    return excessReturns;
}

Conclusion

It may not look very simple, but it actually is straight-forward. We are effectively saying that each user should generate an average comment score of 1 point per day to break even. Anything you make beyond 1 point is considered an excess return for the day. We then simply take the Sharpe ratio of the average daily excess returns. The resulting metric ensures that users are incentivized to make consistent, high-quality submissions and punishes one hit wonders and those who take a spray-and-pray approach.

If you liked this article, please upvote it on HackerNews.

Footnotes

HackerNews

Every founder needs to keep an eye on the startup scene and cutting-edge tools that are being released. I've found competitors, partners, ideas, strategies, tips, and friends via HackerNews. For technologists, it's the best community on the web, period.

Tips:

• Use ihackernews on your mobile phone.

Theme Forest

I wasted a lot of time on designs for EffectCheck before I finally admitted that designing a full website from scratch is a full time job. I wanted to focus on developing an awesome emotional impact analysis engine, not worry about divs and gradients. Theme Forest has tons of awesome themes that you can customize, and at around $15, the price is right!

Tips:

• Search for "CSS3" to find the highest quality site templates.

99Designs

Once you have a site template, you need a logo. This is another area where I tried and failed to come up with something at first. For around $300, you can run a week-long design contest for your logo. I got over 100 submissions and lots of great candidate images.

Tips:

• Provide LOTS of constant feedback. The artists on the site don't really seem to have a passion for your logo, so you need to guide them well.
• Make it blind the last couple of days to prevent copy-cats from flooding in. This also helps encourage the original artists to continue working on your logo.
• Support email is slow. Talk to them on twitter if you have a problem. I tried the email route first, and after 24 hours of no response, I tweeted my complaint-- within 1 hour I had my problem resolved.
• Ask the artists to see lots of color variations. It's trivial for them to change colors and it helps you get a feel for which color schemes you might like on your site.
• Get lots of opinions. Run polls and send links out to friends. Everyone thinks differently; the responses I got from my friends and family varied drastically and helped me decide on a logo. However, be conscious that this is your startup-- so the logo has to look good to you above all else.
• Beware of publicly announcing your polls. I made the mistake of tweeting my poll and got a lot of random people with questionable intentions voting. I'm pretty sure some of them were friends of one of the artists and trying to help him win.

Photoshop and ShutterStock

Even after buying a template and outsourcing your logo design, you will still need some basic skills in Photoshop to get a full website pushed. The most important skill I've found is the ability to take a stock photo and cut it up well. ShutterStock has some great photos at great prices-- I recommend signing up for the cheaper package that lets you have a dozen "medium-sized" downloadable images, since medium usually means 1000x700 or better which is more than enough for site images. The images for the four sliders on the EffectCheck homepage are derived from images on ShutterStock.

Tips:

• Use the magic wand tool to drastically cut down the time to cut images.
• If you're cutting up an image that has a white background, add a new layer and make it a dark color so you can better see the edges of your image. When you save the image, just uncheck that layer.
• Blur the edges of photos just slightly to remove jaggies and make the image look smoother on your page.

.

Git and Github

This almost goes without saying these days, but Git is the clear winner in the battle of the source control versioning systems. Playing off of this, Github has become a social coding explosion-- I've even had a project forked!

Tips:

• Committing lots of open source code to Github is also a great way to boost your startup's network effect among developers.

AppHarbor

As I mentioned in my previous post, my startup is built on .NET and deployed on AppHarbor. My deployment takes seconds, it's hosted on EC2, and it incidentally includes a free private git repo!

Tips:

• If you add non-code files to your project, make sure their properties are set to "content", otherwise they won't be copied to your server instance
• Check out the new service hooks -- pretty awesome!

JQuery and JQuery UI

When you need to add that extra polish to your site, JQuery UI is there. I used it in several places for the EffectCheck editor that our clients use, but also to create the tabbed locations on the contact page.

Tips:

• When debugging AJAX calls, it helps to use a plugin like prettyPrint that will dump the JSON result contents to a pretty-printed table.

StackOverflow

I've asked about 30 questions on StackOverflow since I started working on EffectCheck, but it feels like 300. In fact, if you add in the number of times I found a SO link via Google or started asking a question but found the answer on StackOverflow's similar questions panel, it probably is close to that many. I really believe StackOverflow is one of the most important sites to emerge on the internet in the last five years. It's drastically increased my productivity and knowledge in ways I never thought possible.

Tips:

• Do not be afraid to ask a question. You usually will get an answer within 30 minutes even for tough questions that you've been struggling with for days.
• With that in mind, do not let your questions linger unanswered. Every question you ask should either be marked as answered by someone else or self-answered.

Blogging with WordPress

You need marketing. It may seem like if you build it, they will come-- but they won't. Having an interesting blog and pushing it on social media sites is a good first step. There are lots of choices for blogging engines, but I've found WordPress to be the best.

Tips:

• Use SEO-friendly links with words from the article title in the URL.
• Push social media sites like reddit, Digg, etc. Have a group of friends that will give you a couple upvotes for the initial kick needed to attract the attention of people watching the new stories feed.
• When you are about to hit the front page on a major site, make sure you have a caching plug-in like WP Super Cache setup so you don't get DDoS'd.

Google Apps for Business to Host Email

The free hosting services Google provides for email are great. You can setup groups, create users, and tons of other things all in seconds, plus you get the convenient Gmail web interface.

Tips:

• Setup a group for your sales and development teams (even if they're both just your and your co-founder). It adds an air of legitimacy, especially if your startup is focused on B2B sales.

Postmark for Transactional Emails

If you ever need some way to send emails automatically, like to confirm new user accounts or customer purchases, you have a few options. I like Postmark over Sendgrid or Amazon, though I am hearing a lot of buzz about MailGun. The jury is still out on which app is the best, but EffectCheck uses Postmark and it works great for us!

Tips:

• If you're in B2B sales like EffectCheck, try having a contact form on your site and have Postmark email your sales team whenever someone submits a new entry.

Sublime Text

You need a great text editor. There are too many text files in the world that have a special syntax you need highlighted and whatever comes with your OS is probably not nearly as awesome as Sublime Text. You can even use it for free, though it's more than worth the small fee to buy it!

Tips:

• The auto-complete functionality (ctrl+space on Windows) is particularly nifty, especially if you're coding in a dynamic language like Ruby.
• There's lots of plug-ins available, so don't stop at just the defaults.

Conclusion

That's just a taste of some of the great tools that are out there to help you launch your startup. As I progress, I'm sure I'll find loads more awesome tools worth writing about. Until then, back to hacking together an awesome startup!

Why I chose .NET

I live in Palo Alto, work at the Hacker Dojo, and I am even buying a Macbook Pro tomorrow. Clearly I have totally drank the startup Kool-Aid. The one point I balk at is the idea that choosing .NET is always a mistake. I believe .NET was the right choice for EffectCheck, for the following reasons:

• Existing expertise - I spent three years working as an AI researcher and quant at a hedge fund. We used two languages: R and C#. If you work with any major financial services provider, they invariably have an awesome .NET API and usually a Java one. So I stuck with C# and became really good at it. I do know Ruby/Rails and I'm playing around with Python/Django now to build an iPhone game, but at the end of the day it would take me 5-10x longer to build everything in those languages.
• Core algorithm already implemented - The awesome AI algorithm behind EffectCheck has already been implemented and thoroughly tested in C#. Reimplementing it in another language would mean more testing and debugging.
• AppHarbor - It's Heroku for .NET. You simply can't beat it. I push to my git repo and it compiles and deploys automagically. I know lots of Python hackers who swear up and down that they use EC2 and love it, but the server admin gene just isn't in me. I don't want to spend an hour setting up my AMI to get my web app deployed.
• Visual Studio - Hands down the best programming environment anywhere. I've seen cool things from Textmate with Ruby, but I still think the C# intellisense support is way beyond anything available for other languages/frameworks.
• I know the community - Throughout the last few years, mostly on accident, I've come to know a lot of the best .NET hackers. I did some poker research with Daniel Crenna, I met Aaron Stannard when we both lived in San Diego, and John Sheehan was on my Startup Weekend team. I was also one of the first AppHarbor users (and support tickets!), which means I've had plenty of discussions with Michael Friis, Rune Soerensen, and Troels Thomsen.

Can't we all just get along?

I'm not trying to get into a religious war here. Other stacks have their own benefits. Python has awesome libraries like the nltk, scipy, and numpy; Rails has Heroku and tons of great gems. The .NET choice is just personal preference, but that's kind of my point: it doesn't matter if you use .NET or not.

If you look at ASP.NET MVC 3 compared to the two main MVC frameworks out there (Django and Rails), there's very little conceptual difference. I use 960gs, git, jQuery, and (hopefully) soon MongoDB just like plenty of Rails and Django developers. The rest is just the choice of some arbitrary programming language and MVC framework to route calls.

Conclusion

So should you use .NET? The obvious answer is "it depends." If at least three of my five reasons resonate with you, then I would probably say yes. You really have nothing to fear anymore-- it's all pretty much the same until you reach massive scale. If you reach that point, well, that's one of those good problems.

If you liked this article, please up-vote it on HackerNews.

What is EffectCheck?

EffectCheck is an algorithm that draws on decades of clinical psychology research to identify the subliminal emotional impact of your word choice.

Lots of words have subconscious effects on us. For instance, if you're walking down the street and you see a sign with the word "ELDERLY" written on it, studies¹ show people actually start walking slower without realizing it. Alternatively, consider responding to someone telling a great joke. You can say "That joke is hilarious" or "That joke kills me" -- both mean the same thing, the context is the same, but subconsciously the word "kills" actually raises your stress levels. EffectCheck measures these subliminal effects and helps people discover how their writing likely makes readers feel independent of context.

Who could use it?

Anyone who wants to understand the emotions their readers are going to feel. Some examples:

• Advertisers may want potential customers to feel more confident so the customers buy more merchandise.
• Defense attorneys writing their closing arguments may want the jury to feel compassionate towards the defendant.
• Politicians giving a speech may want to charge the audience by raising anxiety and hostility.

There's clearly a wide range of use cases and EffectCheck can be customized to specific domains given enough training samples.

What will be covered in this series?

I'm new to web development, so I'm hoping this will serve as a semi-roadmap for people looking to build their own startup. I'm specifically focusing on people who have cool algorithms or applications that they've written and want to create a startup around them. Some of the topics I'm planning on covering:

1. Why I chose .NET
2. Running a 99Designs logo contest
3. Building a developer API platform
4. A/B testing, funnel analysis and metrics

There will be a lot more to come also, but that's all I've dealt with so far. With the exception of this intro post and the "Why I chose .NET" post, I'll try to keep everything technical or instructive; I don't want this blog to become about my random musings and opinions. I also will not be covering any of the implementation details behind the EffectCheck algorithm (trade secrets and NDAs). Everything else is fair game, so if you're looking to build a startup, I hope you find this series helpful!

Footnotes

Create your AppHarbor Application

Creating an application on AppHarbor is straight-forward:

1. Go to your Applications tab.
2. Click "Create New".
3. Fill in a name like "TestMembership".
4. Add a database.
5. Choose a MSSQL database with the free 20mb option.
6. Call the database something like "membershipdb".

That's all that you need to do in AppHarbor! Leave the tab with your database info open, you'll need that in a minute.

Create your MVC Application

Create your ASP.NET MVC application like you would normally:

1. In Visual Studio 2010, File -> New -> Project.
2. Create a new ASP.NET MVC 3 Project or MVC 2 if you don't have MVC 3 installed.
3. Choose Internet Application so that the membership provider is setup locally.

This is all pretty standard. Now we have to setup the AppHarbor database with the membership table structure.

Configuring and Using your AppHarbor Database

There is a great utility called aspnet_sqlreg.exe. This tool will setup membership support on your MS SQL database automatically:

1. Start a Visual Studio command prompt by going to Start -> All Programs -> Visual Studio 2010 -> Visual Studio Tools -> Visual Studio Command Prompt.
2. Run the following command, using the your AppHarbor database settings to fill in the values in brackets: aspnet_regsql.exe -S [Host] -d [Name] -U [Username] -P [Password] -A all
3. In your MVC project, open your Web.config file.
4. Replace the connectionString for ApplicationServices with the connection string found on your AppHarbor database page.

This will setup all the membership services on your AppHarbor database, including role management, profile support, etc. The link to the MSDN page on the tool has all the flags in case you only want some features.

Conclusion

That's it! You can now push your app to AppHarbor and test out that your membership code works.

The HyperNEAT algorithm¹ incorporates the geometry of the search space by effectively adding a layer of indirection. Rather than evolving the neural network directly, HyperNEAT evolves a Compositional Pattern Producing Network (CPPN)² which can generate a neural network. CPPNs take the location of two nodes in the problem domain and output the connection between them in the target neural network.

The IGenomeDecoder

The main difference between a HyperNEAT experiment and a standard NEAT experiment in SharpNEAT 2 is the genome decoding. In a NEAT experiment, your CreateGenomeDecoder method simply returns a new NeatGenomeDecoder, and you're done. In a HyperNEAT experiment, you have to create a custom decoder that specifies where the input and output nodes are in the problem domain.

public class TicTacToeHyperNeatExperiment
{
    ...

    /// <summary>
    /// Creates a new HyperNEAT genome decoder that can be used to convert
    /// a genome into a phenome.
    /// </summary>
    public IGenomeDecoder<NeatGenome, IBlackBox> CreateGenomeDecoder()
    {
        // Create an input and an output layer for the HyperNEAT
        // substrate. Each layer corresponds to the game board
        // with 9 squares.
        SubstrateNodeSet inputLayer = new SubstrateNodeSet(9);
        SubstrateNodeSet outputLayer = new SubstrateNodeSet(9);

        // Each node in each layer needs a unique ID.
        // The input nodes use ID range [1,9] and
        // the output nodes use [10,18].
        uint inputId = 1, outputId = 10;

        // The game board is represented as a 2-dimensional plane.
        // Each square is at -1, 0, or 1 in the x and y axis.
        // Thus, the game board looks like this:
        //
        // (-1,1)  | (0,1)   | (1,1)
        // (-1,0)  | (0,0)   | (1,0)
        // (-1,-1) | (0,-1)  | (1,-1)
        //
        // Note that the NeatPlayer class orders the board from
        // left to right, then top to bottom. So we need to create
        // nodes with the following IDs:
        //
        //    1    |    2    |   3
        //    4    |    5    |   6
        //    7    |    8    |   9
        //
        for(int x = -1; x <= 1; x++)
            for (int y = 1; y >= -1; y--, inputId++, outputId++)
            {
                inputLayer.NodeList.Add(new SubstrateNode(inputId, new double[] { x, y }));
                outputLayer.NodeList.Add(new SubstrateNode(outputId, new double[] { x, y }));
            }

        List<SubstrateNodeSet> nodeSetList = new List<SubstrateNodeSet>(2);
        nodeSetList.Add(inputLayer);
        nodeSetList.Add(outputLayer);

        // Define a connection mapping from the input layer to the output layer.
        List<NodeSetMapping> nodeSetMappingList = new List<NodeSetMapping>(1);
        nodeSetMappingList.Add(NodeSetMapping.Create(0, 1, (double?)null));

        // Construct the substrate using a steepened sigmoid as the phenome's
        // activation function. All weights under 0.2 will not generate
        // connections in the final phenome.
        Substrate substrate = new Substrate(nodeSetList,
            DefaultActivationFunctionLibrary.CreateLibraryCppn(),
            0, 0.2, 5, nodeSetMappingList);

        // Create genome decoder. Decodes to a neural network packaged with
        // an activation scheme that defines a fixed number of activations per evaluation.
        IGenomeDecoder<NeatGenome, IBlackBox> genomeDecoder =
            new HyperNeatDecoder(substrate, _activationSchemeCppn,
                                            _activationScheme, false);

        return genomeDecoder;
    }
}

In our experiment, the Tic-Tac-Toe board is our problem domain and the game board is represented as a simple 2-d plane. Each node is given a pair of (x,y) coordinates corresponding to its location on the game board. These coordinates are used as inputs to the CPPN when it's generating the neural network.

The above example is a simple 2-layer CPPN with no hidden layers. If you want to add hidden layers, you simply need to create an additional SubstrateNodeSet and add it to the nodeSetList. Then you will need two NodeSetMappings, one from the input (0) to the hidden (1) layer, and one from the hidden (1) to the output (2) layer.³

Once you have everything specified, you create a Substrate (i.e., the CPPN genome) and return the HyperNeatDecoder.

Saving HyperNEAT Genomes

The only other change is going to be in handling the file I/O for the genomes.

class Program
{
    ...

    static void ea_UpdateEvent(object sender, EventArgs e)
    {
        Console.WriteLine(string.Format("gen={0:N0} bestFitness={1:N6}",
                                _ea.CurrentGeneration, _ea.Statistics._maxFitness));

        // Save the best genome to file
        // Note: Unlike the other experiments, you must save
        // function IDs of HyperNEAT genomes.
        var doc = NeatGenomeXmlIO.SaveComplete(
                                new List() { _ea.CurrentChampGenome }, true);

        doc.Save(CHAMPION_FILE);
    }
}

Since HyperNEAT also evolves each node's activation function, we need to make sure to save that information to file along with the rest of the topology information.

Conclusion

Going from NEAT to HyperNEAT is actually very easy. Once you have a good idea of how you are going to map your problem domain to a geometric plane, it's straight-forward to transition.

Unfortunately, the problems we had still persist: the evolution does not converge on an optimal strategy. Since that was never really the goal of these tutorials, it's okay! Our goal was to learn how to use SharpNEAT 2 to evolve neural networks, which I think I can safely say has been accomplished.

Source code is available here. The project used for this part was TicTacToeHyperNeat.

Creating a Blackjack Game

The basic game rules are well known, but each casino has its own minor variants that can affect the odds and correct plays. The NashCoding Blackjack Framework (NBF) enables simulating a wide variety of these variants via the BlackjackSettings class:

/// <summary>
/// A collection of various settings for a game of Blackjack.
/// </summary>
public class BlackjackSettings
{
    /// <summary>
    /// The number of 52-card decks contained in the dealer's shoe.
    /// </summary>
    public int DecksPerShoe { get; set; }

    /// <summary>
    /// The number of cards to deal before the shoe is reshuffled.
    /// </summary>
    public int MinCardsDealtBeforeReshuffle { get; set; }

    /// <summary>
    /// The minimum amount a player can bet.
    /// </summary>
    public decimal MinimumBet { get; set; }

    /// <summary>
    /// The maximum amount a player can bet.
    /// </summary>
    public decimal MaximumBet { get; set; }

    /// <summary>
    /// The threshold where the dealer stops hitting and
    /// stands (typically 17).
    /// </summary>
    public int DealerHardStandThreshold { get; set; }

    /// <summary>
    /// The threshold where the dealer stops hitting and
    /// stands if one of their cards is a soft ace.
    /// </summary>
    public int DealerSoftStandThreshold { get; set; }

    /// <summary>
    /// The reward multiplier for Blackjack, default is
    /// 2.5 (i.e., pays 1.5:1).
    /// </summary>
    public decimal BlackjackPayoff { get; set; }

    /// <summary>
    /// Whether the player is offered insurance when the
    /// dealer shows a potential blackjack.
    /// </summary>
    public bool InsuranceOffered { get; set; }

    /// <summary>
    /// The cost to purchase insurance, as a fraction
    /// of the original bet.
    /// </summary>
    public decimal InsuranceCost { get; set; }

    /// <summary>
    /// The payoff for insurance, where 2 is 1:1 payoff.
    /// </summary>
    public decimal InsurancePayoff { get; set; }

    /// <summary>
    /// The maximum number of times a player can split
    /// in a hand.
    /// </summary>
    public int MaxSplitsAllowed { get; set; }

    /// <summary>
    /// Whether the player is allowed to resplit aces.
    /// Ex: Dealt AA, split to AA and Ax -- can the
    /// first hand be resplit?
    /// </summary>
    public bool ResplitAcesAllowed { get; set; }

    /// <summary>
    /// Whether the player is allowed to split on 20.
    /// </summary>
    public bool SplitTensAllowed { get; set; }

    /// <summary>
    /// Whether the player is allowed to hit after
    /// splitting aces.
    /// </summary>
    public bool HittingSplitAcesAllowed { get; set; }

    /// <summary>
    /// Whether the player can double down only on ten or
    /// eleven. If false, the player can double down on
    /// any hand.
    /// </summary>
    public bool DoubleDownOnlyTenOrEleven { get; set; }

    /// <summary>
    /// Whether the player is allowed to double down on a
    /// soft hand (contains an ace).
    /// </summary>
    public bool SoftDoubleDownAllowed { get; set; }

    /// <summary>
    /// Whether the player is allowed to double down after
    /// splitting hands other than AA.
    /// </summary>
    public bool DoubleDownNonAceSplitsAllowed { get; set; }

    /// <summary>
    /// Whether the player is allowed to double down after
    /// splitting AA.
    /// </summary>
    public bool DoubleDownSplitAcesAllowed { get; set; }

    /// <summary>
    /// Whether the player is allowed to surrender their hand
    /// and receive a partial payoff back.
    /// </summary>
    public bool SurrenderAllowed { get; set; }

    /// <summary>
    /// The payoff for surrendering, as a fraction of the
    /// original bet. Typically this is 0.5.
    /// </summary>
    public decimal SurrenderPayoff { get; set; }
}

The defined game settings are passed to the constructor of the BlackjackGame engine class:

/// <summary>
/// A Blackjack game engine class. This class enables the user to simulate
/// a potentially-infinite number of blackjack hands played by a collection
/// of players.
/// </summary>
public class BlackjackGame
{
    /// <summary>
    /// The settings for this game.
    /// </summary>
    public BlackjackSettings Settings { get; set; }

    private const int SHUFFLES_PER_SHOE = 10;
    private Shoe _shoe;

    /// <summary>
    /// Creates a new instance of a game of Blackjack.
    /// </summary>
    public BlackjackGame(BlackjackSettings settings)
    {
        Settings = settings;
        _shoe = new Shoe(settings.DecksPerShoe);
    }

    /// <summary>
    /// Plays a game of Blackjack with the collection of players.
    /// </summary>
    public void Play(IEnumerable<IBlackjackPlayer> players)
    {
        ...
    }
}

Blackjack Strategies

Once you've constructed a game, you can then simulate a Blackjack session with any collection of player strategies. The NBF comes with a predefined set of strategies:

• Basic Strategy - This is the standard strategy you find in Las Vegas gift shops on those little playing card-sized action tables. It's a losing strategy, but without card counting it's about as close to a fair bet as you're likely to get.
• Wizard of Odds Simple Strategy - For most people, basic strategy is too complicated to memorize. The Wizard of Odds gives a simplified version of basic strategy that sacrifices a small percentage of expected value compared to basic strategy but is concise enough to memorize.
• Simple Five Count Strategy - The more fives that have been played, the more likely it is the dealer will bust when it hits. This strategy plays basic strategy but counts the number of fives that have been played, increasing the bet size as more fives are dealt.

The three above strategies are just a jumping off point. It's simple enough to create your own strategy that does a more complex counting system, just implement the IBlackjackPlayer interface:

/// <summary>
/// An interface that all Blackjack players and strategies must implement.
/// </summary>
public interface IBlackjackPlayer
{
    /// <summary>
    /// The total amount that the player has won or lost in the current game.
    /// </summary>
    decimal Profit { get; set; }

    /// <summary>
    /// The number of splits this player has done in the current game.
    /// </summary>
    int Splits { get; set; }

    /// <summary>
    /// Returns true if the player will play the next hand.
    /// False means they're done and not sitting back down.
    /// </summary>
    bool PlayAnotherHand();

    /// <summary>
    /// Returns the amount the player is wagering on this hand.
    /// </summary>
    decimal GetBet(decimal min, decimal max);

    /// <summary>
    /// Returns true if the player wants to split the hand.
    /// </summary>
    bool Split(HandInfo info);

    /// <summary>
    /// Returns true if the player wants to double down on the hand.
    /// </summary>
    bool DoubleDown(HandInfo info);

    /// <summary>
    /// Returns true if the player wants to hit the hand.
    /// </summary>
    bool Hit(HandInfo info);

    /// <summary>
    /// Returns true if the player wants to buy insurance.
    /// </summary>
    bool BuyInsurance(HandInfo info);

    /// <summary>
    /// Returns true if the player wants to surrender this hand.
    /// </summary>
    bool Surrender(HandInfo info);

    /// <summary>
    /// Updates the player with the full info on the hand and how
    /// much they won or lost on it. Counting strategies can update
    /// here.
    /// </summary>
    void HandOver(HandInfo info);

    /// <summary>
    /// Called when the dealer shuffles the deck
    /// </summary>
    void Reshuffle();
}

Running the Simulator

The ConsoleBlackjack project provides the skeleton needed to run the project:

class Program
{
    static void Main(string[] args)
    {
        BlackjackSettings settings = LoadSettingsFromFile("settings.xml");
        BlackjackGame game = new BlackjackGame(settings);

        var handsToPlay = 100000000L;
        var player = new SimpleFiveCountPlayer(handsToPlay);

        game.Play(new [] { player });
        Console.WriteLine("Profit: {0:N2}%",
                player.Profit / settings.MinimumBet / (decimal)handsToPlay * 100m);
    }

    ...
}

Note that this project also provides a ConsoleBlackjackPlayer that enables you to play blackjack yourself via the command line. Just create a new instance of ConsoleBlackjackPlayer the same way you would any other strategy, and set the Game property to the Blackjack game instance.

Conclusion

The NashCoding Blackjack Framework is designed to facilitate statistics and AI research in the game of Blackjack. It should be able to simulate about 100,000 hands per second on a modern machine, and typically you need at least 100 million hands to get a stable expected value for a strategy. Although the game has been researched heavily, there are still a lot of questions to be answered particularly regarding the interaction of multiple strategies. If you find the framework useful or if you discover a bug, please let me know.

Source code for this post is available here.

Champion parameter values using squared losses as fitness.

As you can see, the fitness function does actually find the NE for the AKQ game. It stabilizes at player 1 betting 1/3 of the time with a queen and player 2 calling 1/3 of the time with a king.

The current SharpNEAT release does not include any support for coevolution, so we're going to build it from scratch.²

The ICoevolutionPhenomeEvaluator

If you recall from Part 1, we created a custom implementation of the IPhenomeEvaluator. This evaluator played a black box against both a random player and an optimal player then returned a fitness score. We'll stick to the same design by creating the ICoevolutionPhenomeEvaluator:

/// <summary>
/// Represents an evaluator that competes two phenomes against each other.
/// </summary>
public interface ICoevolutionPhenomeEvaluator<TPhenome>
{
    ...
    /// <summary>
    /// Evaluate the provided phenomes and return their fitness scores.
    /// </summary>
    void Evaluate(TPhenome phenome1, TPhenome phenome2,
                       out FitnessInfo fitness1, out FitnessInfo fitness2);
}

Classes implementing this interface define an Evaluate method that competes phenome1 against against phenome2, setting their respective fitness scores at the end.

A Parallel Coevolution Genome List Evaluator

The standard approach in two-player competitive coevolution is to play round-robin style (i.e., exhaustively compete every possible pair of individuals). Scores are then aggregated together into a final fitness score for each individual. To do this, we need to create a custom implementation of IGenomeListEvaluator.

public class ParallelCoevolutionListEvaluator<TGenome, TPhenome>
      : IGenomeListEvaluator<TGenome>
{
    readonly IGenomeDecoder<TGenome, TPhenome> _genomeDecoder;
    readonly ICoevolutionPhenomeEvaluator<TPhenome> _phenomeEvaluator;
    readonly ParallelOptions _parallelOptions;
    ...
    /// <summary>
    /// Main genome evaluation loop with no phenome caching (decode
    /// on each evaluation). Individuals are competed pairwise against
    /// every other in the population.
    /// Evaluations are summed to get the final genome fitness.
    /// </summary>
    public void Evaluate(IList<TGenome> genomeList)
    {
        //Create a temporary list of fitness values
        FitnessInfo[] results = new FitnessInfo[genomeList.Count];
        for (int i = 0; i < results.Length; i++)
            results[i] = FitnessInfo.Zero;

        // Exhaustively compete individuals against each other.
        Parallel.For(0, genomeList.Count, delegate(int i)
        {
            for(int j = 0; j < genomeList.Count; j++)
            {
                // Don't bother evaluating inviduals against themselves.
                if (i == j)
                    continue;

                // Decode the first genome.
                TPhenome phenome1 = _genomeDecoder.Decode(genomeList[i]);

                // Check that the first genome is valid.
                if (phenome1 == null)
                    continue;

                // Decode the second genome.
                TPhenome phenome2 = _genomeDecoder.Decode(genomeList[j]);

                // Check that the second genome is valid.
                if (phenome2 == null)
                    continue;

                // Compete the two individuals against each other and get
                // the results.
                FitnessInfo fitness1, fitness2;
                _phenomeEvaluator.Evaluate(phenome1, phenome2,
                                                       out fitness1, out fitness2);

                // Add the results to each genome's overall fitness.
                // Note that we need to use a lock here because
                // the += operation is not atomic.
                lock (results)
                {
                    results[i]._fitness += fitness1._fitness;
                    results[i]._alternativeFitness +=
                                                  fitness1._alternativeFitness;
                    results[j]._fitness += fitness2._fitness;
                    results[j]._alternativeFitness +=
                                                  fitness2._alternativeFitness;
                }
            }
        });

        // Update every genome in the population with its new fitness score.
        for (int i = 0; i < results.Length; i++)
        {
            genomeList[i].EvaluationInfo.SetFitness(results[i]._fitness);
            genomeList[i].EvaluationInfo.AlternativeFitness =
                                                  results[i]._alternativeFitness;
        }
    }
}

We start by creating an array of fitness scores that will store the overall fitness of each genome. We then do a parallel nested for-loop over the population.³ Each individual is decoded from a genome to its phenotypic representation, and the two phenomes are evaluated against each other. Once the evaluation is finished, we update the genomes' overall fitness scores. When all evaluations are done, we set the genome's final fitness scores.

We've now built a general two-player competitive coevolution extension for SharpNEAT. It's time to get back to our specific problem domain.

Creating a Coevolution Evaluator for Tic-Tac-Toe

Our coevolution evaluator will take two black boxes and compete them in a game of Tic-Tac-Toe, with 10 points for a win, 1 point for a draw, and 0 points for a loss.⁴ The code is actually pretty simple:

public class TicTacToeCoevolutionEvaluator : ICoevolutionPhenomeEvaluator
{
    ...

    /// <summary>
    /// Evaluate the two black boxes by playing them against each other in a
    /// game of Tic-Tac-Toe. All permutations of size 2 are going to be
    /// evaluated, so we only play one game: box1 is X, box2 is O.
    /// </summary>
    public void Evaluate(IBlackBox box1, IBlackBox box2,
                        out FitnessInfo fitness1, out FitnessInfo fitness2)
    {
        // box1 plays as X.
        NeatPlayer player1 = new NeatPlayer(box1, SquareTypes.X);

        // box2 plays as O.
        NeatPlayer player2 = new NeatPlayer(box2, SquareTypes.O);

        // Play the two boxes against each other.
        var winner = TicTacToeGame.PlayGameToEnd(player1, player2);

        // Score box1
        double score1 = getScore(winner, SquareTypes.X);
        fitness1 = new FitnessInfo(score1, score1);

        // Score box2
        double score2 = getScore(winner, SquareTypes.O);
        fitness2 = new FitnessInfo(score2, score2);

        // Update the evaluation counter.
        _evalCount++;
    }
}

Our ParallelCoevolutionListEvaluator is going to evaluate all permutations, so we will see matchups of genome #1 vs. genome #2 and genome #2 vs. genome #1. Thus, we only have to play 1 game per evaluation, with player 1 as X.

A Custom Experiment

Since we're using our own list evaluator, we will need to create our own INeatExperiment. Most of the code is identical to the code from SimpleNeatExperiment in part 1. The only important difference is in the CreateEvolutionAlgorithm method:

public class TicTacToeCoevolutionExperiment : INeatExperiment
{
    ...

    /// <summary>
    /// Create and return a NeatEvolutionAlgorithm object ready for running
    /// the NEAT algorithm/search. Various sub-parts of the algorithm are
    /// also constructed and connected up.
    /// This overload accepts a pre-built genome2 population and their
    /// associated/parent genome factory.
    /// </summary>
    public NeatEvolutionAlgorithm<NeatGenome> CreateEvolutionAlgorithm(
             IGenomeFactory<NeatGenome> genomeFactory,
             List<NeatGenome> genomeList)
    {
        // Create distance metric. Mismatched genes have a fixed distance
        // of 10; for matched genes the distance is their weigth difference.
        IDistanceMetric distanceMetric = new ManhattanDistanceMetric(1.0, 0.0, 10.0);
        ISpeciationStrategy<NeatGenome> speciationStrategy =
                  new ParallelKMeansClusteringStrategy<NeatGenome>(
                      distanceMetric, _parallelOptions);

        // Create complexity regulation strategy.
        IComplexityRegulationStrategy complexityRegulationStrategy =
                  ExperimentUtils.CreateComplexityRegulationStrategy(
                     _complexityRegulationStr, _complexityThreshold);

        // Create the evolution algorithm.
        NeatEvolutionAlgorithm<NeatGenome> ea =
                  new NeatEvolutionAlgorithm<NeatGenome>(
                    _eaParams, speciationStrategy, complexityRegulationStrategy);

        // Create genome decoder.
        IGenomeDecoder<NeatGenome, IBlackBox> genomeDecoder =
                  CreateGenomeDecoder();

        // Create a genome list evaluator. This packages up the genome decoder
        // with the phenome evaluator.
        IGenomeListEvaluator<NeatGenome> genomeListEvaluator =
                  new ParallelCoevolutionListEvaluator<NeatGenome, IBlackBox>(
                      genomeDecoder, PhenomeEvaluator);

        // Wrap a hall of fame evaluator around the baseline evaluator.
        genomeListEvaluator =
                 new ParallelHallOfFameListEvaluator<NeatGenome, IBlackBox>(
                     50, 0.5, ea, genomeListEvaluator, genomeDecoder, PhenomeEvaluator);

        // Initialize the evolution algorithm.
        ea.Initialize(genomeListEvaluator, genomeFactory, genomeList);

        // Finished. Return the evolution algorithm
        return ea;
    }
    ...
}

The first portion of the code sets up the way SharpNEAT performs speciation, complexity regulation, and the basic EvolutionAlgorithm. We aren't toying with those parts (maybe in a future tutorial).

After that, we get to the code that creates our custom list evaluator. If you compare this with the code for SimpleNeatExperiment, you'll see all we had to do here was replace type of evaluator we're creating.

Bonus! A Hall of Fame Evaluator

In the above code you probably noticed we are also creating something called ParallelHallOfFameListEvaluator and passing it our original list evaluator. It's common in coevolution experiments to maintain a "hall of fame" where every few generations we save the best individual. Future individuals then have to compete against not just the current population but also the best individuals from the previous populations. Let's take a look at the first part of how this evaluator is implemented:

/// <summary>
/// Represents a Hall of Fame evaluator.
///
/// Champion genomes are periodically stored to create a Hall of Fame. All successive
/// generations are required to evaluate against both their own population and
/// the hall of fame. A genome's overall fitness is a weighted sum of its
/// fitness against the population and the hall of fame.
/// </summary>
public class ParallelHallOfFameListEvaluator<TGenome, TPhenome>
      : IGenomeListEvaluator<TGenome>
{
    readonly IGenomeDecoder<TGenome, TPhenome> _genomeDecoder;
    readonly ICoevolutionPhenomeEvaluator<TPhenome> _phenomeEvaluator;
    readonly ParallelOptions _parallelOptions;
    readonly uint _generationsPerChampion;
    readonly IGenomeListEvaluator<TGenome> _innerEvaluator;
    readonly double _hallOfFameWeight;

    uint _lastUpdate;
    List<TGenome> _hallOfFame;

    ...

    /// <summary>
    /// Construct with the provided number of generations per champion, weight to the hall
    /// of fame fitness, parallel options, and other parameters.
    /// </summary>
    public ParallelHallOfFameListEvaluator(uint generationsPerChampion,
                              double hallOfFameWeight,
                              AbstractGenerationalAlgorithm<TGenome> ea,
                              IGenomeListEvaluator<TGenome> innerEvaluator,
                              IGenomeDecoder<TGenome, TPhenome> genomeDecoder,
                              ICoevolutionPhenomeEvaluator<TPhenome> phenomeEvaluator,
                              ParallelOptions options)
    {
        Debug.Assert(hallOfFameWeight >= 0d);
        Debug.Assert(hallOfFameWeight <= 1d);

        _generationsPerChampion = generationsPerChampion;
        _hallOfFameWeight = hallOfFameWeight;
        _innerEvaluator = innerEvaluator;
        _genomeDecoder = genomeDecoder;
        _phenomeEvaluator = phenomeEvaluator;
        _parallelOptions = options;

        _hallOfFame = new List<TGenome>();
        ea.UpdateEvent += new EventHandler(ea_UpdateEvent);
    }

    ...

    private void ea_UpdateEvent(object sender, EventArgs e)
    {
        // Make sure that the event sender is an EA.
        Debug.Assert(sender is AbstractGenerationalAlgorithm<TGenome>);

        // Cast the EA so we can access the current champion.
        AbstractGenerationalAlgorithm<TGenome> ea =
                         (AbstractGenerationalAlgorithm<TGenome>)sender;

        // Update every few generations.
        if (ea.CurrentGeneration < (_generationsPerChampion + _lastUpdate))
            return;

        // Update the update counter.
        _lastUpdate = ea.CurrentGeneration;

        // Add the genome to the hall of fame.
        _hallOfFame.Add(ea.CurrentChampGenome);

        Console.WriteLine("Hall of Fame Updated. Size: {0}", _hallOfFame.Count);
    }

}

The hall of fame evaluator takes a few parameters worth noting:

• generationsPerChampion - The number of generations that have to elapse before saving another population champion to the hall of fame.
• hallOfFameWeight - The weight given to the fitness against the hall of fame, as a percentage of the total weight. For example, if hallOfFameWeight is 0.3, then the final fitness for a genome will be 0.3 * hallOfFameFitness + 0.7 * innerEvaluatorFitness.
• innerEvaluator - The evaluator to use as the other portion of the overall fitness.

In the constructor, the hall of fame evaluator adds itself to the evolution algorithm's update event. The EA will trigger the ea_UpdateEvent method periodically, according to whatever UpdateScheme we have defined. We assume that whatever scheme the user has chosen is what they want, and we work around it in ea_UpdateEvent. In this method, we check if we are due for an update and if we are, we add the current population champion to the hall of fame.

Most of the evaluation code is identical to the coevolution list evaluator:

public class ParallelHallOfFameListEvaluator<TGenome, TPhenome>
      : IGenomeListEvaluator<TGenome>
{
    ...

    /// <summary>
    /// Main genome evaluation loop with no phenome caching (decode on each evaluation).
    /// Individuals are competed pairwise against every champion in the hall of fame.
    /// The final fitness score is the weighted sum of the fitness versus the champions
    /// and the fitness score by the inner evaluator.
    /// </summary>
    public void Evaluate(IList<TGenome> genomeList)
    {
        _innerEvaluator.Evaluate(genomeList);

        // Create a temporary list of fitness values
        // with the scores of the inner evaluator.
        FitnessInfo[] results = new FitnessInfo[genomeList.Count];
        double innerWeight = 1.0 - _hallOfFameWeight;
        for (int i = 0; i < results.Length; i++)
            results[i] = new FitnessInfo(
                        genomeList[i].EvaluationInfo.Fitness * innerWeight,
                        genomeList[i].EvaluationInfo.AlternativeFitness * innerWeight);

        // Calculate how much each champion game is worth
        double championGameWeight = _hallOfFameWeight / (double)_hallOfFame.Count;

        // Exhaustively compete individuals against each other.
        Parallel.For(0, genomeList.Count, delegate(int i)
        {
            // Decode the first genome.
            TPhenome phenome1 = _genomeDecoder.Decode(genomeList[i]);

            // Check that the first genome is valid.
            if (phenome1 == null)
                return;

            for (int j = 0; j < _hallOfFame.Count; j++)
            {
                // Decode the second genome.
                TPhenome phenome2 = _genomeDecoder.Decode(_hallOfFame[j]);

                // Check that the second genome is valid.
                if (phenome2 == null)
                    continue;

                // Compete the two individuals against each other and get the results.
                FitnessInfo fitness1, fitness2;
                _phenomeEvaluator.Evaluate(phenome1, phenome2,
                                                       out fitness1, out fitness2);

                // Add the results to each genome's overall fitness.
                // Note that we need to use a lock here because
                // the += operation is not atomic.
                lock (results)
                {
                    results[i]._fitness += fitness1._fitness * championGameWeight;
                    results[i]._alternativeFitness +=
                                  fitness1._alternativeFitness * championGameWeight;
                }
            }
        });

        // Update every genome in the population with its new fitness score.
        for (int i = 0; i < results.Length; i++)
        {
            genomeList[i].EvaluationInfo.SetFitness(results[i]._fitness);
            genomeList[i].EvaluationInfo.AlternativeFitness =
                                                          results[i]._alternativeFitness;
        }
    }

    ...
}

There are a couple differences from the coevolution list evaluator. Rather than setting the overall fitness scores to zero at the start, we set them equal to the weighted value of the inner evaluator's fitness scores. Also, our inner loop is over the hall of fame rather than the population. The rest of code then functions just like the coevolution evaluator, but adds weighted scores for each individual evaluation.

Running the Experiment

Surprisingly, our actual main Program file is almost exactly the same. The only alteration we have to make is that we're creating a different type of experiment (TicTacToeCoevolutionExperiment):

class Program
{
    static NeatEvolutionAlgorithm<NeatGenome> _ea;
    const string CHAMPION_FILE =
                 @"..\..\..\NeatTicTacToe\bin\Debug\coevolution_champion.xml";

    static void Main(string[] args)
    {
        // Initialise log4net (log to console).
        XmlConfigurator.Configure(new FileInfo("log4net.properties"));

        // Experiment classes encapsulate much of the nuts
       // and bolts of setting up a NEAT search.
        TicTacToeCoevolutionExperiment experiment =
                                      new TicTacToeCoevolutionExperiment();

        // Load config XML.
        XmlDocument xmlConfig = new XmlDocument();
        xmlConfig.Load("tictactoe.config.xml");
        experiment.Initialize("TicTacToe", xmlConfig.DocumentElement);

        // Create evolution algorithm and attach update event.
        _ea = experiment.CreateEvolutionAlgorithm();
        _ea.UpdateEvent += new EventHandler(ea_UpdateEvent);

        // Start algorithm (it will run on a background thread).
        _ea.StartContinue();

        // Hit return to quit.
        Console.ReadLine();
    }

    static void ea_UpdateEvent(object sender, EventArgs e)
    {
        Console.WriteLine(string.Format("gen={0:N0} bestFitness={1:N6}",
                        _ea.CurrentGeneration, _ea.Statistics._maxFitness));

        // Save the best genome to file
        var doc = NeatGenomeXmlIO.SaveComplete(
                       new List<NeatGenome>() { _ea.CurrentChampGenome }, false);
        doc.Save(CHAMPION_FILE);
    }
}

The program saves the best genome to the GUI's debug\bin directory, so you can launch the game and easily load the champion XML file.

Conclusion

That's it! In this tutorial, we saw how to create a two-player competitive coevolution experiment in SharpNEAT. We created an experiment to evolve a Tic-Tac-Toe AI by playing individuals in the population them against each other. By now you know how to setup a coevolution experiment, create custom population evaluators, and handle update events

If you played against our AI, you probably noticed it (still) sucks. That's (still) okay! The coevolution player is definitely at least as good as the player from part 1, and we didn't have to hand-code any opponent strategies, so I'm calling it a success. The next part of this tutorial will introduce HyperNEAT, a powerful extension to the NEAT algorithm that enables us to leverage the inherent geometry and symmetry of the game board. The result will (hopefully) be a stronger AI opponent.

Source code for part 2 of the tutorial is available here.

Update July 25, 2010: I implemented a second approach to competitive coevolution. This approach, called Host-Parasite coevolution, has two populations evolving simultaneously. Fitness scoring is then a function of performance against the other population (called the parasite population) and a hall of fame of previous generational champions. The project doesn't cover any new features of SharpNEAT, so I'm not going to make a full write up on it. If you want to read more about the approach see this paper. Updated source code is available here.

Footnotes