Commit Graph

65 Commits

Author SHA1 Message Date
4ce124fdab Fix flaky TestNomadStderrFifoIsRemoved
that expected only one allocation per job. However, in practice, a job can be rescheduled on multiple allocations.
2023-11-17 23:38:06 +00:00
41eb2627ae Handle stopping goroutines of the MemoryLeakTestSuite.
We observed a race condition that sometimes goroutines are just closing but still measured by the goroutine Lookup of the following test.
We try to avoid this by adding `runtime.Gosched()` at the start of each test.
2023-11-16 12:10:57 +01:00
2ea24911de Fix inconsistent calculation of goroutines.
We observed that sometimes the `pprof.Lookup("goroutine")` resulted in a different goroutine count than `runtime.NumGoroutine()`. It might be due to a race condition, but our usage of both made it impossible to debug.
2023-11-16 12:10:57 +01:00
6b69a2d732 Refactor Nomad Recovery
from an approach that loaded the runners only once at the startup
to a method that will be repeated i.e. if the Nomad Event Stream connection interrupts.
2023-10-31 15:49:56 +01:00
3abd4d9a3d Refactor all tests to use the MemoryLeakTestSuite. 2023-09-11 13:44:29 +02:00
a01bd0fa7e Provide Memory Leak Test Suite
by adding an assertion about the number of Goroutines to the unit tests.
2023-09-11 13:44:29 +02:00
0fd6e42487 Add regression e2e test for incomplete debug message.
See #325.
2023-08-14 11:37:51 +02:00
5b64725faa Fix golangci-lint errors
that appeared due to the new version v1.53.1.
2023-06-04 11:54:42 +01:00
0218f91cd4 Disable AWS e2e tests. 2023-03-14 23:42:19 +01:00
e0419c2e58 Fix Sentry Debug Regex
that was ignoring composed messages including a newline.
Also, add regression test.
2023-03-14 23:42:19 +01:00
7dadc5dfe9 Refactor Nomad Command Generation.
- Abstracting from the exec form while generating.
- Removal of single quotes (usage of only double-quotes).
- Bash-nesting using escaping of special characters.
2023-03-14 23:42:19 +01:00
0b7f71f3dc #190 Add recovery e2e tests. 2022-11-26 13:33:44 +00:00
0c6c48c3cf #190 Add unit tests for runner recovery. 2022-11-26 13:33:44 +00:00
4c25473c9e Hide Nomad specific environment variables
from the user environment.
2022-11-04 15:15:43 +01:00
b3eee17846 Support protected directories
by setting the sticky bit to all explicitly requested directories.
2022-10-29 19:11:05 +02:00
939f36dac6 Fix e2e test exhausted cpu
by decreasing the reservation amount.
2022-10-26 01:27:33 +02:00
3e9606d1b9 Fix flaky e2e tests because of no idle runners
at startup.
2022-10-14 21:40:09 +01:00
195f88177e Add Content-Length and Content-Disposition Header
for GetFileContent route.
2022-10-05 12:11:47 +01:00
847e5cda65 Extend ls2json reader
by also parsing the link target, permissions, group and owner.
2022-10-05 12:11:47 +01:00
3469d0ce77 Specify http not found exit code
by replacing it with StatusGone (410) for a missing runner and StatusFailedDependency (424) for missing or not accessible files.
2022-10-05 12:11:47 +01:00
152b77afe5 Add listing of runners file system. 2022-10-05 12:11:47 +01:00
f2b25566dd #136 Copy files back from Nomad runner. 2022-10-05 12:11:47 +01:00
1a5a49d7c8 Explicitly switch user for code execution.
Co-authored-by: Maximilian Pass <maximilian.pass@student.hpi.uni-potsdam.de>
2022-09-24 23:09:23 +02:00
7d58d2b99c CI Debug Print AWS functions 2022-08-01 16:52:34 +02:00
0094776a88 CI Debug WIP 2022-08-01 14:39:12 +02:00
d0ad9e03a3 Suppress request noctx linter. 2022-08-01 14:35:52 +02:00
ed5205236f #166 Fix AWS JAVA ignoring shell commands in combination with a Makefile. 2022-06-21 13:24:01 +02:00
b0916a2485 #166 Adjust outdated make command pattern
as with #147 a new unset command part is introduced.

Also add a regression e2e test that would have caught this issue.
2022-06-21 13:24:01 +02:00
2f066a9fb4 E2e test environment variables. 2022-06-10 18:10:28 +02:00
0f8a1fa25a Specify AWS Functions as list
to conform with the yaml standard of list definition.
2022-06-08 09:01:46 +02:00
97a2311a74 Fix CodeClimate duplication smells
by extracting one http helper method and
increasing the threshold for duplication issues.
2022-06-07 16:59:02 +02:00
a41659eed4 Enable memory oversubscription (#102)
* Enable memory oversubscription

* Fix and add e2e test
2022-03-18 08:31:27 +01:00
4ffbb712ed Parametrize e2e tests to also check AWS environments.
- Fix destroy runner after timeout.
- Add file deletion
2022-02-28 14:54:40 +01:00
f3929cbf50 Fix webSocket error
that was introduced by closing the WebSocket connection without stopping the inputLoop.
2021-12-16 15:05:47 +01:00
f45d28d4a4 Fix flaky TestOutputToStdout test 2021-12-12 10:27:03 +01:00
825ebdd3e6 Add forcePull option
* Add forcePull option
for pulling the image when the execution environment gets updated

* Apply suggestions from code review

Co-authored-by: Sebastian Serth <MrSerth@users.noreply.github.com>

* Add unit tests

* Clean up and implement option two

Co-authored-by: Sebastian Serth <MrSerth@users.noreply.github.com>
2021-12-09 14:54:14 +01:00
24b22f2c40 Fix racing condition
when creating a Nomad job in the e2e tests
2021-10-22 16:23:51 +02:00
34d4bb7ea0 Implement routes to list, get and delete execution environments
* #9 Implement routes to list, get and delete execution environments.
A refactoring was required to introduce the ExecutionEnvironment interface.

* Fix MR comments, linting issues and bug that lead to e2e test failure

* Add e2e tests

* Add unit tests
2021-10-21 10:33:52 +02:00
9b106f4cd8 Fix linting issues
An update of golangci-lint yielded new linting issues. This commit
fixes them.
2021-08-05 13:40:48 +02:00
c8c5357b8c Rename module for GitHub 2021-07-30 16:43:05 +02:00
6a60b6cd89 Add config option to enable (m)TLS between Poseidon and Nomad 2021-07-29 09:43:21 +00:00
91537a7364 Use test docker image in e2e tests
The TestCreateOrUpdateEnvironment function would previously use
the python:latest Docker image in its execution environment request.
However, this lead to pull rate limiting by Docker Hub in our CI.
2021-07-27 15:26:53 +00:00
8b26ecbe5f Restructure project
We previously didn't really had any structure in our project apart
from creating a new folder for each package in our project root.
Now that we have accumulated some packages, we use the well-known
Golang project layout in order to clearly communicate our intent
with packages. See https://github.com/golang-standards/project-layout
2021-07-21 12:55:35 +02:00
2f1383b743 Add tests for returning mapped ports of runners 2021-07-21 08:22:10 +02:00
c7606f3d5f Fix a lot of linting issues
After we introduced the linter we haven't really touched the old code.
This commit now fixes all linting issue that exist right now.
2021-07-13 08:59:25 +02:00
bd7fb53385 Fix bug that the count of the default task group is set to the prewarming pool size 2021-07-07 09:21:57 +02:00
68eacae7fe Fix bug that config task group is not added to the template job (and the faulty tests) 2021-07-06 10:09:36 +02:00
bbc1ce12ca Delete idle runners when the environment is scaled down 2021-07-02 13:00:13 +02:00
ae08e37106 Add end to end test for inactivity timeout 2021-06-23 11:04:19 +02:00
c7ed54942d Move ChannelReceivesSomething to tests package.
ChannelReceivesSomething (formerly WaitForChannel) originally was
located in the helpers package.
This move was done to remove a cyclic dependency with the nomand package.
2021-06-21 10:54:07 +02:00