Designing for the Future of Code Tests

My week involved a little bit of actually writing code--getting a new branch working and a couple of bug fixes--but a lot of design. Earlier in the week, I had a meeting with Kyle to continue a discussion on the way Code Tests are currently stored on the server, what changes need to be made to accomidate the new Code Tests design, as well as what systems we will want to have in place for future potential developements.

A New Code Test Protocol

For code tests in their current state, not much needed to be altered to the code_tests table in the database:

• crawl_cd: The column that previously stored an enum value for the AST node type crawled will have an expanded enum for the new options that can be crawled.
• return_cd: An additional column was added, similar to the crawl_cd, to store an enum for the AST node type that is returned. I'm planning on have both these columns reference the same enum, rather than two separate enums that represent the same object type.
• crawler_bool1 (2 & 3): Three columns were added to hold boolean values for crawler options. These vary by crawler type, but in the current design (as well the future options currently still on my To-Do list), no crawler has more than 3 boolean options. I'm a little bit tempted to add a fourth boolean column, just in case there's an option later that we discover seems very necessary, but obviously any numerical choice is going to lock in the design, so this seems a little nebulous to me.

Future Feature: Code Test Helper Methods

There were two other ideas discussed at our meeting that we wanted to ensure were supported by the table structure, even if they were not implemented for a while (so that the database design could be finalized).

The first was calling Code Tests from other Code Tests. For example, say a user writes an awesome Code Test to determine if a user has used a DoTogether construct correctly. Then later, they want to see if there is a WhileLoop that contains a good DoTogether. Why shouldn't they be able to call the previous test they wrote as a "helper method" for their new test?

On the database side, based on some of the ideas Kyle had thought about, I think the best option is to create a new table, code_tests_references, that contains the ID of the calling Code Test (in my example, the While-DoTogether test) and the ID of the helper Code Test (eg. the good DoTogether test). Then, when a Code Test gets loaded by the server, the dependent helper Code Tests can be located by referencing this table. Storing the information in this manner allows Code Tests to be executed as a single transaction (since the helper tests won't need to be located at runtime).

I haven't fully fleshed out how this will look on the IDE side yet, since a lot of design questions quickly appear. At this point, I think I will provide users with a button to "Add helper test..." which will then automaticall add Jython code that stores this helper test as a variable. Then the user could call the test, for example, by writing:

isDoTogetherGood = helpertest1.isValidConstruct(myDoTogether)

The questions that are raised mostly relate how locating these helper tests would go. Since we've made the decision to only save Code Tests on the server (no local save), will a user only have access to the tests that they have created, or can they have access to others' tests, too? If one mentor wrote the "ultimate" DoTogether check, shouldn't other mentors have access to it? But then how will other mentors find this test, or compare it to the other DoTogether check tests? And are users allowed to create tests that exist ONLY as helper tests? 

Future Feature: Versioning

The other discussion Kyle and I had was about an idea of versioning for tests: perhaps a mentor publishes a test, but later thinks of some error to fix or improvement. Perhaps they will want to make a "new version" of the same test. My plan for keeping track of this involves one existing (but repurposed) field plus one new field in the code_tests table:

• published: In one domain, this field will keep track of whether a code test has simply been saved as a draft, or whether it should be published and run on worlds. However, I also picture that there's a third value this can take: deprecated. A deprecated code test remains on the server, but is not executed. Instead, it has a...
• parent_id: Kyle thought we might want to remove this field, but I imagine that, for deprecated tests, this refers to the newer version of the test. In this way, all the previous versions of a test could be constructed by a series of queries to find the parents of a current test. The downside to this is that for a test with many previous versions, this would lead to a ton of queries.
  
  An alternative would be to have an extra field version that would track the version of the test, and have all parent_id entries point to the same original test. This has the advantage that all versions of a test could be found with one query, but will take up more space, even for tests that have a single version. Which version we use will depend on whether we think that most tests will have many successive versions that we all want to have info on at once, or if we think most tests will only have two or three versions each.

A New Composite Design

Lastly, during some Git downtime this week (for some reason, I did some fetches that hung for awhile), I took a first stab at redesigning the Code Test composite window. Right now, the window is gigantic (almost as large as the rest of the IDE screen) and contains a ton of info and options. The main idea of the redesign is to group the info and options into groups, and put those groups on separate tabs. The four tabs I currently sketched up are:

• Info: Give the test a name and description, or load a test you've previously saved
• Options: Choose your crawler type and customize it.
• Write and Test: Write the code for the test and see the results when run on the current world.
• Review: Review the test you've written, and either save it as a draft or publish it.

I think this design helps show the user only the information that is relevant to them at each step of the test design process, and I've worked on a presentation that would keep everything in a small, unobtrusive column to the side, so that the user can easily see both the Code Test Composite and the code at the same time.

I won't have time to do much more design or implementation work on this until I settle the Code Test uploading, and the write the remaining outstanding feature necessary for badges, but it's nice to have some ideas in my back pocket.