Commit Graph

34 Commits

Author SHA1 Message Date
Maximilian Paß
a41659eed4 Enable memory oversubscription (#102)
* Enable memory oversubscription

* Fix and add e2e test
2022-03-18 08:31:27 +01:00
Maximilian Paß
4ffbb712ed Parametrize e2e tests to also check AWS environments.
- Fix destroy runner after timeout.
- Add file deletion
2022-02-28 14:54:40 +01:00
Maximilian Paß
f3929cbf50 Fix webSocket error
that was introduced by closing the WebSocket connection without stopping the inputLoop.
2021-12-16 15:05:47 +01:00
Maximilian Paß
f45d28d4a4 Fix flaky TestOutputToStdout test 2021-12-12 10:27:03 +01:00
Maximilian Paß
825ebdd3e6 Add forcePull option
* Add forcePull option
for pulling the image when the execution environment gets updated

* Apply suggestions from code review

Co-authored-by: Sebastian Serth <MrSerth@users.noreply.github.com>

* Add unit tests

* Clean up and implement option two

Co-authored-by: Sebastian Serth <MrSerth@users.noreply.github.com>
2021-12-09 14:54:14 +01:00
Maximilian Paß
24b22f2c40 Fix racing condition
when creating a Nomad job in the e2e tests
2021-10-22 16:23:51 +02:00
Maximilian Paß
34d4bb7ea0 Implement routes to list, get and delete execution environments
* #9 Implement routes to list, get and delete execution environments.
A refactoring was required to introduce the ExecutionEnvironment interface.

* Fix MR comments, linting issues and bug that lead to e2e test failure

* Add e2e tests

* Add unit tests
2021-10-21 10:33:52 +02:00
sirkrypt0
9b106f4cd8 Fix linting issues
An update of golangci-lint yielded new linting issues. This commit
fixes them.
2021-08-05 13:40:48 +02:00
Maximilian Paß
c8c5357b8c Rename module for GitHub 2021-07-30 16:43:05 +02:00
Jan-Eric Hellenberg
6a60b6cd89 Add config option to enable (m)TLS between Poseidon and Nomad 2021-07-29 09:43:21 +00:00
sirkrypt0
91537a7364 Use test docker image in e2e tests
The TestCreateOrUpdateEnvironment function would previously use
the python:latest Docker image in its execution environment request.
However, this lead to pull rate limiting by Docker Hub in our CI.
2021-07-27 15:26:53 +00:00
sirkrypt0
8b26ecbe5f Restructure project
We previously didn't really had any structure in our project apart
from creating a new folder for each package in our project root.
Now that we have accumulated some packages, we use the well-known
Golang project layout in order to clearly communicate our intent
with packages. See https://github.com/golang-standards/project-layout
2021-07-21 12:55:35 +02:00
sirkrypt0
2f1383b743 Add tests for returning mapped ports of runners 2021-07-21 08:22:10 +02:00
sirkrypt0
c7606f3d5f Fix a lot of linting issues
After we introduced the linter we haven't really touched the old code.
This commit now fixes all linting issue that exist right now.
2021-07-13 08:59:25 +02:00
Maximilian Paß
bd7fb53385 Fix bug that the count of the default task group is set to the prewarming pool size 2021-07-07 09:21:57 +02:00
Maximilian Paß
68eacae7fe Fix bug that config task group is not added to the template job (and the faulty tests) 2021-07-06 10:09:36 +02:00
Maximilian Paß
bbc1ce12ca Delete idle runners when the environment is scaled down 2021-07-02 13:00:13 +02:00
Konrad Hanff
ae08e37106 Add end to end test for inactivity timeout 2021-06-23 11:04:19 +02:00
Konrad Hanff
c7ed54942d Move ChannelReceivesSomething to tests package.
ChannelReceivesSomething (formerly WaitForChannel) originally was
located in the helpers package.
This move was done to remove a cyclic dependency with the nomand package.
2021-06-21 10:54:07 +02:00
Konrad Hanff
92f1af83ae Add tests for codeOceanToRaw and null readers
The tests ensure the readers do not return when there is no data
available.
2021-06-21 08:20:04 +00:00
Maximilian Paß
2e4a975588 Implement even more merge request comments 2021-06-15 12:05:51 +02:00
Maximilian Paß
87f823756b Implement merge request comments 2021-06-15 11:37:47 +02:00
Maximilian Paß
25d78df557 Restore existing jobs and fix rebase (7c99eff3) issues 2021-06-15 11:37:35 +02:00
sirkrypt0
8de489929e Remove stderr fifo after interactive execution with stderr finished
Previously the stderr fifo would not be removed, leaving unwanted
artifacts from the execution behind. We now remove the stderr fifo
after the command finished.
2021-06-14 15:04:09 +02:00
sirkrypt0
d3300e839e Add unit tests for separate stdout and stderr on execution 2021-06-11 08:47:25 +00:00
sirkrypt0
f122dd9376 Split stdout and stderr on interactive execution
When running a command interactively, we previously would get stdout
and stderr both served on stdout by Nomad. To circumvent this issue,
we now start a separate execution inside the allocation to split
both streams.
2021-06-11 08:47:25 +00:00
Jan-Eric Hellenberg
61bc7d0143 Add unit tests for provide runner route 2021-06-10 06:11:31 +00:00
Maximilian Paß
32fe47d669 Implement linting issues and merge request comments 2021-06-09 08:35:20 +00:00
Jan-Eric Hellenberg
ce2b82d43d Copy files with relative path to active workspace directory of container 2021-06-09 10:24:29 +02:00
sirkrypt0
1c4daa99a9 Add e2e tests for exec env createOrUpdate
This also adds a Nomad client to the e2e_tests that can be used to
query Nomad and validate that certain actions happened in Nomad
correctly.
2021-06-03 13:21:49 +00:00
sirkrypt0
b990df7b9d Add route to create or update execution environments 2021-06-03 13:21:49 +00:00
Jan-Eric Hellenberg
02b3f52a11 Add ability to copy files to and delete files from runner 2021-06-02 14:54:54 +02:00
Konrad Hanff
242d0175a2 Add tests and end-to-end tests for websocket execution
For unit tests, this mocks the runners Execute method with a
customizable function that operates on the request, streams and exit
channel to simulate a real execution.

End-to-end tests are moved to the tests/e2e_tests folder. The tests
folder allows us to have shared helper functions for all tests in a
separate package (tests) that is not included in the non-test build.

This also adds one second of delay before each end-to-end test case by
using the TestSetup method of suite. By slowing down test execution,
this gives Nomad time to create new allocations when a test requested a
runner. Another solution could be to increase the scale of the job to
have enough allocations for all end-to-end tests.

Co-authored-by: Maximilian Paß <maximilian.pass@student.hpi.uni-potsdam.de>
2021-05-31 12:32:51 +02:00
Jan-Eric Hellenberg
619cd40fb6 Refactor EntityStore interface to multiple interfaces of specific type 2021-05-27 09:56:07 +00:00