Plat_Forms - The web development platform comparison

Plat_Forms

Welcome to the official website of Plat_Forms - The web development platform comparison

"Plat_Forms" is a contest and competition in which top-class teams of three programmers compete to implement the same requirements for a web-based system within two days, using different technology platforms (e.g. Java, .NET, Perl, PHP, Python, Ruby, Scala, Smalltalk, JavaScript or what-have-you).

Its purpose is not to determine "the" best platform, but rather to provide new insights into the real (rather than purported) pros, cons, and emergent properties of each platform. The evaluation will analyze many aspects of each solution, both external (scalability, functionality, reliability, security, performance, etc.) and internal (structure, modularity, understandability, flexibility, etc.).

Plat_Forms 2012 is over. We will keep you up to date with the evaluation progress. The task document is now available for the general public at The Task 2012.

What has been going on? - part 3

Submitted by ustaerk on Mon, 2013-04-15 11:38
Plat_Forms: 
Plat_Forms 2012 rev. 2

Today we will conclude our little series of preliminary results. After the robustness data in part 1 and the activity data in part 2, today we will show you the completeness of our participants' solutions.

Completeness

For each requirement from the requirements document two randomly assigned judges evaluated a solution's implementation of that requirement and categorized it as missing, partially implemented, fully-implemented-but-flawed (e.g. failing in over 25% of the use cases or being buggy), fully implemented, and particularly-well-fully-implemented. Figure 1 shows all fully implemented requirements, i.e. omits missing and only partially implemented requirements.

Figure 1: Number of fully implemented requirements by requirement priority and team. The rightmost bar for comparison shows the possible maximum.

What has been going on? - part 2

Submitted by ustaerk on Fri, 2013-04-05 16:17
Plat_Forms: 
Plat_Forms 2012 rev. 2

As promised last week, today we share another bit of data with you.

Activity interviews

During the contest every 15 minutes we asked each team member what he is currently doing. We listened to all 2328 interviews thus collected and labeled the answers with codes corresponding to one of several activity types. Figure 1 shows the totals (per code) for each team.

Figure 1: Frequency of each type of activity performed at the time of our interviews.

What has been going on?

Submitted by ustaerk on Fri, 2013-04-05 15:06
Plat_Forms: 
Plat_Forms 2012 rev. 2

It has been a while since Plat_Forms 2012 and today we want to share with you what we have been doing in the meantime.

We have collected an enormous amount of data (interviews during the contest, code, version archives, running solutions, questionnaires, and more) and haven't finished our evaluation yet. In particular we sofar cannot draw any platform-specific conclusions. As you might remember, Plat_Forms 2012 wanted to find a new platform notion that is not tied to a particular programming language but based on characteristics of the technology stack. This turns out to be a difficult task that we are still working on.

In the following three weeks we will therefore simply show a few per-team charts without comment.

What we do have is data on the type of activity the teams were performing at the moment of our interviews (every 15 minutes), data on the completeness of the teams' solutions, and data on the robustness of the solutions.

What's missing and currently being worked on amongst others is data on the maintainability of the solutions, data on the static structure of the solutions (several notation-specific scanners still need to be written), and of course the attempt to explain the observations with the technology employed. So stay tuned for updates.

Robustness

For our robustness tests we stressed the solutions with different uncommon inputs -- some valid, others invalid. Figure 1 shows our observations.

Figure 1: Results of our robustness tests. Light red stands for broken, yellow for acceptable and green for OK. We submitted input with HTML tags ("</...>" column), veeeeeery long character strings ("long" column), Unicode characters ("int'l" column), 5 different types of invalid email addresses ("email" column), strings containing SQL quote characters ("SQL" column), and we disabled cookies before trying to login ("cookie" column).