Commit Graph

34 Commits

Author SHA1 Message Date
a41659eed4 Enable memory oversubscription (#102)
* Enable memory oversubscription

* Fix and add e2e test
2022-03-18 08:31:27 +01:00
4ffbb712ed Parametrize e2e tests to also check AWS environments.
- Fix destroy runner after timeout.
- Add file deletion
2022-02-28 14:54:40 +01:00
f3929cbf50 Fix webSocket error
that was introduced by closing the WebSocket connection without stopping the inputLoop.
2021-12-16 15:05:47 +01:00
f45d28d4a4 Fix flaky TestOutputToStdout test 2021-12-12 10:27:03 +01:00
825ebdd3e6 Add forcePull option
* Add forcePull option
for pulling the image when the execution environment gets updated

* Apply suggestions from code review

Co-authored-by: Sebastian Serth <MrSerth@users.noreply.github.com>

* Add unit tests

* Clean up and implement option two

Co-authored-by: Sebastian Serth <MrSerth@users.noreply.github.com>
2021-12-09 14:54:14 +01:00
24b22f2c40 Fix racing condition
when creating a Nomad job in the e2e tests
2021-10-22 16:23:51 +02:00
34d4bb7ea0 Implement routes to list, get and delete execution environments
* #9 Implement routes to list, get and delete execution environments.
A refactoring was required to introduce the ExecutionEnvironment interface.

* Fix MR comments, linting issues and bug that lead to e2e test failure

* Add e2e tests

* Add unit tests
2021-10-21 10:33:52 +02:00
9b106f4cd8 Fix linting issues
An update of golangci-lint yielded new linting issues. This commit
fixes them.
2021-08-05 13:40:48 +02:00
c8c5357b8c Rename module for GitHub 2021-07-30 16:43:05 +02:00
6a60b6cd89 Add config option to enable (m)TLS between Poseidon and Nomad 2021-07-29 09:43:21 +00:00
91537a7364 Use test docker image in e2e tests
The TestCreateOrUpdateEnvironment function would previously use
the python:latest Docker image in its execution environment request.
However, this lead to pull rate limiting by Docker Hub in our CI.
2021-07-27 15:26:53 +00:00
8b26ecbe5f Restructure project
We previously didn't really had any structure in our project apart
from creating a new folder for each package in our project root.
Now that we have accumulated some packages, we use the well-known
Golang project layout in order to clearly communicate our intent
with packages. See https://github.com/golang-standards/project-layout
2021-07-21 12:55:35 +02:00
2f1383b743 Add tests for returning mapped ports of runners 2021-07-21 08:22:10 +02:00
c7606f3d5f Fix a lot of linting issues
After we introduced the linter we haven't really touched the old code.
This commit now fixes all linting issue that exist right now.
2021-07-13 08:59:25 +02:00
bd7fb53385 Fix bug that the count of the default task group is set to the prewarming pool size 2021-07-07 09:21:57 +02:00
68eacae7fe Fix bug that config task group is not added to the template job (and the faulty tests) 2021-07-06 10:09:36 +02:00
bbc1ce12ca Delete idle runners when the environment is scaled down 2021-07-02 13:00:13 +02:00
ae08e37106 Add end to end test for inactivity timeout 2021-06-23 11:04:19 +02:00
c7ed54942d Move ChannelReceivesSomething to tests package.
ChannelReceivesSomething (formerly WaitForChannel) originally was
located in the helpers package.
This move was done to remove a cyclic dependency with the nomand package.
2021-06-21 10:54:07 +02:00
92f1af83ae Add tests for codeOceanToRaw and null readers
The tests ensure the readers do not return when there is no data
available.
2021-06-21 08:20:04 +00:00
2e4a975588 Implement even more merge request comments 2021-06-15 12:05:51 +02:00
87f823756b Implement merge request comments 2021-06-15 11:37:47 +02:00
25d78df557 Restore existing jobs and fix rebase (7c99eff3) issues 2021-06-15 11:37:35 +02:00
8de489929e Remove stderr fifo after interactive execution with stderr finished
Previously the stderr fifo would not be removed, leaving unwanted
artifacts from the execution behind. We now remove the stderr fifo
after the command finished.
2021-06-14 15:04:09 +02:00
d3300e839e Add unit tests for separate stdout and stderr on execution 2021-06-11 08:47:25 +00:00
f122dd9376 Split stdout and stderr on interactive execution
When running a command interactively, we previously would get stdout
and stderr both served on stdout by Nomad. To circumvent this issue,
we now start a separate execution inside the allocation to split
both streams.
2021-06-11 08:47:25 +00:00
61bc7d0143 Add unit tests for provide runner route 2021-06-10 06:11:31 +00:00
32fe47d669 Implement linting issues and merge request comments 2021-06-09 08:35:20 +00:00
ce2b82d43d Copy files with relative path to active workspace directory of container 2021-06-09 10:24:29 +02:00
1c4daa99a9 Add e2e tests for exec env createOrUpdate
This also adds a Nomad client to the e2e_tests that can be used to
query Nomad and validate that certain actions happened in Nomad
correctly.
2021-06-03 13:21:49 +00:00
b990df7b9d Add route to create or update execution environments 2021-06-03 13:21:49 +00:00
02b3f52a11 Add ability to copy files to and delete files from runner 2021-06-02 14:54:54 +02:00
242d0175a2 Add tests and end-to-end tests for websocket execution
For unit tests, this mocks the runners Execute method with a
customizable function that operates on the request, streams and exit
channel to simulate a real execution.

End-to-end tests are moved to the tests/e2e_tests folder. The tests
folder allows us to have shared helper functions for all tests in a
separate package (tests) that is not included in the non-test build.

This also adds one second of delay before each end-to-end test case by
using the TestSetup method of suite. By slowing down test execution,
this gives Nomad time to create new allocations when a test requested a
runner. Another solution could be to increase the scale of the job to
have enough allocations for all end-to-end tests.

Co-authored-by: Maximilian Paß <maximilian.pass@student.hpi.uni-potsdam.de>
2021-05-31 12:32:51 +02:00
619cd40fb6 Refactor EntityStore interface to multiple interfaces of specific type 2021-05-27 09:56:07 +00:00