Unit Tests and Headless Execution

This week I continued to work on preparing to take the Code Test code to the server. I believe that now, we are arguably in a good place to begin working on that (more about "arguably" in a second).

The main change I made that now allows Code Tests to run (again) on the server was to make the code test execution, once again, headless. One of the changes I had introduced to the runtime was feedback for the user when they're writing tests in the IDE. This was for three reasons:

1. It allowed the user to include print statements in their Jython code for debugging purposes
2. When there was a Python/Jython error, it allowed the user to see the error messages and correct their tests.
3. It allowed feedback on the state of the test ("Ready", "Running...", "Done", or "Failed"). This prevented a scenario where a test crashed from being identical to a scenario when a test ran but found no results.

Unfortunately, this change meant that the runtime was communicating with a Croquet component (the Test Editor window), which does not exist on the server, so the execution would fail. To correct this, I utilized an idea that Kyle and I had talked about several weeks ago: create a new class that the Runtime writes to, and the Test Editor window reads from. So now, the runtime and the editor never communicate directly to each other. In execution on the server, the runtime can still write to this new class, and the fact that there is no Croquet component reading from it does not matter. (I can even perhaps think of implications in the future where we'd want to see what error codes users are getting if their tests are failing, so this class may have a secondary use).

In addition, I also rounded out a set of 19 unit tests that check Code Tests for expected execution of correct tests, and expectedly interrupted execution for incorrect or dangerous tests. The good news is that 18 out of these 19 tests are running perfectly, exactly as expected.

The not awful news is that the one test that's failing, although a very crucial test--that users aren't allowed to read/write to files on the server--is failing because the preLaunch() utility function isn't about to execute from the JUnit tests, and this is what sets up the permissions and security manager. When trying to read/write to files from the IDE code tests window, behavior fails as expected (because preLaunch() has run). I had a hard time trying to get preLaunch() to run from the JUnit tests, I believe because JUnit tests have their own permissions that limit what can be accessed, although I'm not positive that's what's happening. So the bottom line is--if we can accept that the SecurityManager unit test fails because preLaunch() isn't running, but in situations where preLaunch() does run, that test would pass, I feel like we're in good shape to start turning our attention server-side.