Add new provision script, Vagrantfile and update local setup guide
This commit is contained in:
Felix Auringer
committed by
Sebastian Serth
Sebastian Serth
parent
281813c3d1
commit
667ab52504
14
Vagrantfile
vendored
14
Vagrantfile
vendored
@ -2,13 +2,13 @@
|
||||
# vi: set ft=ruby :
|
||||
|
||||
Vagrant.configure(2) do |config|
|
||||
config.vm.box = "ubuntu/bionic64"
|
||||
config.vm.box = "ubuntu/focal64"
|
||||
config.vm.provider "virtualbox" do |v|
|
||||
v.memory = 8192
|
||||
v.cpus = 2
|
||||
v.memory = 4096
|
||||
v.cpus = 4
|
||||
end
|
||||
config.vm.network "private_network", ip: "192.168.59.104"
|
||||
config.vm.network "forwarded_port", guest: 3035, host: 3035
|
||||
# config.vm.synced_folder "../data", "/vagrant_data"
|
||||
config.vm.provision "shell", path: "provision.sh", privileged: false
|
||||
config.vm.network "forwarded_port", host_ip: "127.0.0.1", host: 3000, guest: 3000
|
||||
config.vm.synced_folder ".", "/home/vagrant/codeocean"
|
||||
config.vm.synced_folder "../dockercontainerpool", "/home/vagrant/dockercontainerpool"
|
||||
config.vm.provision "shell", path: "provision/provision.vagrant.sh", privileged: false
|
||||
end
|
||||
|
||||
@ -4,6 +4,7 @@ default: &default
|
||||
password: CHANGE_ME
|
||||
pool: 16
|
||||
username: postgres
|
||||
host: localhost
|
||||
|
||||
development:
|
||||
<<: *default
|
||||
|
||||
@ -14,7 +14,7 @@ development:
|
||||
workspace_root: <%= Rails.root.join('tmp', 'files', Rails.env) %>
|
||||
pool:
|
||||
active: true
|
||||
location: http://localhost:3000
|
||||
location: http://localhost:3100
|
||||
refill:
|
||||
async: false
|
||||
batch_size: 8
|
||||
|
||||
@ -1,72 +1,66 @@
|
||||
# Local Setup
|
||||
|
||||
If available, we prefer a native setup for best performance and less technical issues. Please see below for some details.
|
||||
|
||||
|
||||
## Vagrant
|
||||
|
||||
## Install prerequisites
|
||||
Install Vagrant - https://www.vagrantup.com/docs/installation/
|
||||
Install VirtualBox - https://www.virtualbox.org/wiki/Downloads
|
||||
### Install prerequisites
|
||||
|
||||
## Install and setup
|
||||
### Clone repository
|
||||
Create a local codeOceanRoot: mkdir /path/to/CodeOcean ==> codeOceanRoot = /path/to/CodeOcean
|
||||
Clone Repository (https://github.com/openHPI/codeocean) to codeOceanRoot
|
||||
cd codeOceanRoot
|
||||
- [Install Vagrant](https://www.vagrantup.com/docs/installation)
|
||||
- [Install VirtualBox](https://www.virtualbox.org/wiki/Downloads)
|
||||
|
||||
### Get Vagrant base image
|
||||
vagrant box add ubuntu/trusty64
|
||||
vagrant up
|
||||
### Clone repositories
|
||||
|
||||
### Trouble shooting
|
||||
(sometimes, particularly if VirtualBox is running under Windows as the host sysstem, parts of the provision script are) not executed.
|
||||
vagrant up does not show error messages but later on the trouble starts.
|
||||
The following two repositories have to be cloned in the same directory:
|
||||
|
||||
ln -s /etc/nginx/sites-available/code_ocean /etc/nginx/sites-enabled <= Failed (no such directory)
|
||||
- [CodeOcean](https://github.com/openHPI/codeocean)
|
||||
- [DockerContainerPool](https://github.com/openHPI/dockercontainerpool)
|
||||
|
||||
#### Make docker daemon useable without sudo
|
||||
Infos taken from: http://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo
|
||||
Vagrant assumes that these repositories are completely clean. For example, Vagrant will setup all configuration files in `config` (in both repositories) based on the examples provided in the same directory. Therefore it is **important** that these configuration files do not exist before running vagrant up. It is recommended to have a freshly cloned repository but you can also try to remove untracked files by running `git clean -xf` in both repositories.
|
||||
|
||||
vagrant ssh
|
||||
sudo groupadd docker
|
||||
sudo gpasswd -a ${USER} docker
|
||||
sudo service docker restart
|
||||
newgrp docker
|
||||
### Create and start VM
|
||||
|
||||
apt-get install nginx
|
||||
ln -s /etc/nginx/sites-available/code_ocean /etc/nginx/sites-enabled
|
||||
|
||||
#### If ruby version needs to be updated (as provision.sh is not up-to-date :( )
|
||||
Infos taken from: http://stackoverflow.com/questions/26242712/installing-rvm-getting-error-there-was-an-error23
|
||||
|
||||
vagrant ssh
|
||||
rvm group add rvm "$USER"
|
||||
|
||||
logout and login again
|
||||
rvm fix-permissions (not necessarily required)
|
||||
rvm install (requested ruby version)
|
||||
|
||||
cd /vagrant
|
||||
gem install bundler
|
||||
bundle install
|
||||
|
||||
#### Pending migrations
|
||||
vagrant ssh
|
||||
cd /vagrant
|
||||
rake db:migrate
|
||||
|
||||
#### Missing config files or anything else goes wrong
|
||||
Check the according parts of the provision.sh file and try to re-run them directly in the vagrant VM.
|
||||
All problems that have occurred resulted from a more restrictive rights management in the VMs that run under a Windows host system.
|
||||
- Switch to the `codeocean` directory
|
||||
- Run `vagrant up`
|
||||
- If this command fails please try the following:
|
||||
- Run `vagrant destroy -f` to remove the broken VM
|
||||
- Make sure that both repositories are freshly cloned, for example by deleting and cloning them again
|
||||
- Retry to execute `vagrant up`
|
||||
- The VM pulls only one docker image: [`openhpi/co_execenv_python:3.8`](https://hub.docker.com/layers/openhpi/co_execenv_python/3.8/images/sha256-b048f61d490d1b202016dc3bdf99a5169ec998109ae9bbae441c94bdec18e3d0)
|
||||
|
||||
### Start server
|
||||
vagrant ssh
|
||||
cd /vagrant
|
||||
rails s -p 3000 -b 0.0.0.0
|
||||
|
||||
### Login to CodeOcean
|
||||
192.168.59.104:3000
|
||||
admin@example.org:admin
|
||||
You can [configure vagrant as remote interpreter in RubyMine](https://www.jetbrains.com/help/ruby/configuring-language-interpreter.html#add_remote_ruby_interpreter) and start the rails server via RubyMine or you can start it manually from the command line:
|
||||
|
||||
```bash
|
||||
vagrant ssh
|
||||
cd /home/vagrant/dockercontainerpool
|
||||
rails s -p 3100
|
||||
|
||||
# using another ssh session
|
||||
cd /home/vagrant/codeocean
|
||||
rails s -p 3000 -b 0.0.0.0
|
||||
```
|
||||
|
||||
The default credentials for the administrator are:
|
||||
|
||||
- email: `admin@example.org`
|
||||
- password: `admin`
|
||||
|
||||
## Execution Environments
|
||||
|
||||
Every exercise is executed in an execution environment which is based on a docker image. In order to install a new image, have a look at the container of the openHPI team on [DockerHub](https://hub.docker.com/u/openhpi). For example you can add an [image for ruby](https://hub.docker.com/layers/openhpi/co_execenv_ruby/latest/images/sha256-70f597320567678bf8d0146d93fb1bd98457abe61c3b642e832d4e4fbe7f4526) by executing `docker pull openhpi/co_execenv_ruby:latest`.
|
||||
After that make sure to configure the corresponding execution environment for the docker images you want to use in your CodeOcean instance. Therefore sign in on your running CodeOcean server as an administrator and select `Execution Environments` from the `Administration` dropdown. The `Docker Container Pool Size` should be greater than 0 for every execution environment you want to use.
|
||||
|
||||
## Webpack
|
||||
|
||||
This project uses `webpacker` to integrate Webpack with Rails to deliver Frontend assets. During development, the `webpack-dev-server` automatically launches together with the Rails server if not specified otherwise. In case of missing JavaScript or stylesheets or for hot reloading in the browser, you might want to start the `webpack-dev-server` manually *before starting Rails*:
|
||||
|
||||
```shell script
|
||||
./bin/webpack-dev-server
|
||||
```
|
||||
|
||||
This will launch a dedicated server on port 3035 (default setting) and allow incoming WebSocket connections from your browser.
|
||||
|
||||
## Native setup (for macOS)
|
||||
|
||||
@ -165,13 +159,3 @@ admin@example.org:admin
|
||||
```shell script
|
||||
rails s
|
||||
```
|
||||
|
||||
## Webpack
|
||||
|
||||
This project uses `webpacker` to integrate Webpack with Rails to deliver Frontend assets. During development, the `webpack-dev-server` automatically launches togehter with the Rails server if not specified otherwise. In case of missing JavaScript or stylesheets or for hot reloading in the browser, you might want to start the `webpack-dev-server` manually *before starting Rails*:
|
||||
|
||||
```shell script
|
||||
./bin/webpack-dev-server
|
||||
```
|
||||
|
||||
This will launch a dedicated server on port 3035 (default setting) and allow incoming WebSocket connections from your browser.
|
||||
|
||||
168
provision.sh
168
provision.sh
@ -1,168 +0,0 @@
|
||||
#!/bin/bash
|
||||
# rvm/rails installation from https://gorails.com/setup/ubuntu/14.04
|
||||
# passenger installation from https://www.phusionpassenger.com/library/install/nginx/install/oss/trusty/
|
||||
|
||||
|
||||
echo "OUTDATED - This script might be outdated due to the dedicated DockerContainerPool"
|
||||
|
||||
######## VERSION INFORMATION ########
|
||||
|
||||
postgres_version=10
|
||||
ruby_version=2.6.3
|
||||
rails_version=5.2.3
|
||||
geckodriver_version=0.26.0
|
||||
|
||||
########## INSTALL SCRIPT ###########
|
||||
|
||||
# PostgreSQL
|
||||
sudo add-apt-repository "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -sc)-pgdg main"
|
||||
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
|
||||
|
||||
# passenger
|
||||
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
|
||||
sudo apt-get -qq -y install apt-transport-https ca-certificates
|
||||
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main > /etc/apt/sources.list.d/passenger.list'
|
||||
|
||||
# yarn & node
|
||||
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
|
||||
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
|
||||
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
|
||||
|
||||
sudo apt-get -qq update
|
||||
|
||||
# code_ocean
|
||||
sudo apt-get -qq -y install postgresql-client postgresql-$postgres_version postgresql-server-dev-$postgres_version vagrant yarn nodejs
|
||||
|
||||
# Docker
|
||||
if [ ! -f /etc/default/docker ]
|
||||
then
|
||||
# If Docker doesn't start, check this article
|
||||
# https://medium.com/@dirk.avery/docker-error-response-from-daemon-1d46235ff61d
|
||||
curl -sSL https://get.docker.com/ | sudo sh
|
||||
fi
|
||||
if ! grep code_ocean /etc/default/docker
|
||||
then
|
||||
sudo tee -a /etc/default/docker <<EOF
|
||||
|
||||
# code_ocean: enable TCP
|
||||
DOCKER_OPTS="-H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock"
|
||||
EOF
|
||||
sudo service docker restart
|
||||
fi
|
||||
|
||||
# run docker without sudo
|
||||
sudo gpasswd -a ${USER} docker
|
||||
sudo service docker restart
|
||||
|
||||
sudo docker pull openhpi/docker_java
|
||||
sudo docker pull openhpi/docker_ruby
|
||||
sudo docker pull openhpi/docker_python
|
||||
sudo docker pull openhpi/co_execenv_python
|
||||
sudo docker pull openhpi/co_execenv_node
|
||||
sudo docker pull openhpi/co_execenv_java
|
||||
sudo docker pull openhpi/co_execenv_java_antlr
|
||||
sudo docker pull openhpi/co_execenv_ruby
|
||||
sudo docker pull openhpi/co_execenv_ruby:2.5
|
||||
|
||||
# rvm
|
||||
sudo apt-get -qq -y install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev libgdbm-dev libncurses5-dev automake libtool bison libffi-dev
|
||||
gpg --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3
|
||||
curl -sSL https://get.rvm.io | sudo bash -s stable
|
||||
|
||||
# access rvm installation without sudo
|
||||
sudo gpasswd -a ${USER} rvm
|
||||
|
||||
# ruby
|
||||
source /etc/profile.d/rvm.sh
|
||||
sg rvm "rvm install $ruby_version"
|
||||
rvm use $ruby_version --default
|
||||
sudo /usr/local/rvm/bin/rvm alias create default $ruby_version
|
||||
ruby -v
|
||||
|
||||
# rails
|
||||
sg rvm "/usr/local/rvm/rubies/ruby-$ruby_version/bin/gem install rails -v $rails_version"
|
||||
# sudo gem install bundler
|
||||
|
||||
# drop postgres access control
|
||||
if ! sudo grep -q code_ocean /etc/postgresql/$postgres_version/main/pg_hba.conf
|
||||
then
|
||||
sudo tee /etc/postgresql/$postgres_version/main/pg_hba.conf <<EOF
|
||||
# code_ocean: drop access control
|
||||
local all all trust
|
||||
host all all 127.0.0.1/32 trust
|
||||
host all all ::1/128 trust
|
||||
EOF
|
||||
sudo service postgresql restart
|
||||
fi
|
||||
|
||||
# create database
|
||||
if ! (sudo -u postgres psql -l | grep -q code_ocean_development)
|
||||
then
|
||||
sudo -u postgres createdb code_ocean_development || true
|
||||
fi
|
||||
if ! (sudo -u postgres psql -l | grep -q code_ocean_test)
|
||||
then
|
||||
sudo -u postgres createdb code_ocean_test || true
|
||||
fi
|
||||
|
||||
# Selenium tests
|
||||
sudo apt-get -qq -y install xvfb firefox
|
||||
wget --quiet -O ~/geckodriverdownload.tar.gz https://github.com/mozilla/geckodriver/releases/download/v$geckodriver_version/geckodriver-v$geckodriver_version-linux64.tar.gz
|
||||
sudo tar -xzf ~/geckodriverdownload.tar.gz -C /usr/local/bin
|
||||
rm ~/geckodriverdownload.tar.gz
|
||||
sudo chmod +x /usr/local/bin/geckodriver
|
||||
|
||||
# nginx and passenger
|
||||
sudo apt-get -qq -y install nginx-extras passenger
|
||||
|
||||
############# codeocean install ###########################
|
||||
cd /vagrant
|
||||
|
||||
# config
|
||||
for f in action_mailer.yml database.yml secrets.yml sendmail.yml smtp.yml code_ocean.yml docker.yml.erb mnemosyne.yml
|
||||
do
|
||||
if [ ! -f config/$f ]
|
||||
then
|
||||
cp config/$f.example config/$f
|
||||
fi
|
||||
done
|
||||
|
||||
# install code
|
||||
sg rvm 'bundle install'
|
||||
|
||||
# create database
|
||||
export RAILS_ENV=development
|
||||
rake db:schema:load
|
||||
rake db:migrate
|
||||
sg docker 'rake db:seed'
|
||||
sudo mkdir -p /shared
|
||||
sudo chown -R vagrant /shared
|
||||
ln -sf /shared tmp/files #make sure you are running vagrant with admin privileges
|
||||
|
||||
# NGINX
|
||||
if [ ! -L /etc/nginx/sites-enabled/code_ocean ]
|
||||
then
|
||||
sudo tee /etc/nginx/sites-available/code_ocean <<EOF
|
||||
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
|
||||
server {
|
||||
server_name codeocean.local;
|
||||
root /vagrant/public;
|
||||
passenger_ruby /usr/local/rvm/gems/ruby-$ruby_version/wrappers/ruby;
|
||||
passenger_sticky_sessions on;
|
||||
passenger_enabled on;
|
||||
passenger_app_env development;
|
||||
}
|
||||
EOF
|
||||
sudo rm -f /etc/nginx/sites-enabled/default
|
||||
sudo ln -s /etc/nginx/sites-available/code_ocean /etc/nginx/sites-enabled
|
||||
#sudo service nginx restart
|
||||
#cd /vagrant/ && rails s
|
||||
fi
|
||||
|
||||
# Always set language to English
|
||||
sudo locale-gen en_US en_US.UTF-8
|
||||
|
||||
# Add host as alias for localhost (allows sending a score to a local Xikolo instance)
|
||||
sudo tee /etc/hosts -a <<EOF
|
||||
192.168.59.1 localhost
|
||||
EOF
|
||||
112
provision/provision.vagrant.sh
Normal file
112
provision/provision.vagrant.sh
Normal file
@ -0,0 +1,112 @@
|
||||
#!/bin/bash
|
||||
|
||||
######## VERSION INFORMATION ########
|
||||
|
||||
postgres_version=13
|
||||
node_version=14
|
||||
ruby_version=2.7.2
|
||||
|
||||
########## INSTALL SCRIPT ###########
|
||||
|
||||
DISTRO="$(lsb_release -cs)"
|
||||
|
||||
# Prerequisites
|
||||
sudo apt -qq update
|
||||
sudo apt -qq -y install apt-transport-https ca-certificates curl gnupg-agent software-properties-common firefox firefox-geckodriver libpq-dev
|
||||
sudo apt -qq -y upgrade
|
||||
|
||||
# PostgreSQL
|
||||
curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
|
||||
echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list
|
||||
sudo apt -qq update && sudo apt -qq install -y postgresql-client postgresql
|
||||
|
||||
sudo sed -i "/# TYPE/q" /etc/postgresql/$postgres_version/main/pg_hba.conf
|
||||
sudo tee -a /etc/postgresql/$postgres_version/main/pg_hba.conf <<EOF
|
||||
# code_ocean: drop access control
|
||||
local all all trust
|
||||
host all all 127.0.0.1/32 trust
|
||||
host all all ::1/128 trust
|
||||
EOF
|
||||
sudo systemctl restart postgresql
|
||||
|
||||
# Install node
|
||||
curl -sSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | sudo apt-key add -
|
||||
echo "deb https://deb.nodesource.com/node_$node_version.x $DISTRO main" | sudo tee /etc/apt/sources.list.d/nodesource.list
|
||||
echo "deb-src https://deb.nodesource.com/node_$node_version.x $DISTRO main" | sudo tee -a /etc/apt/sources.list.d/nodesource.list
|
||||
sudo apt -qq update && sudo apt -qq install -y nodejs
|
||||
|
||||
# yarn
|
||||
curl -sSL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
|
||||
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
|
||||
sudo apt -qq update && sudo apt -qq install -y yarn
|
||||
|
||||
# Docker
|
||||
curl -sSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
|
||||
sudo add-apt-repository \
|
||||
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
|
||||
$DISTRO \
|
||||
stable"
|
||||
sudo apt -qq update && sudo apt -qq -y install docker-ce docker-ce-cli containerd.io
|
||||
|
||||
sudo usermod -aG docker ${USER}
|
||||
|
||||
sudo tee -a /etc/docker/daemon.json <<EOF
|
||||
{
|
||||
"userns-remap": "default"
|
||||
}
|
||||
EOF
|
||||
|
||||
sudo mkdir -p /etc/systemd/system/docker.service.d/
|
||||
sudo tee -a /etc/systemd/system/docker.service.d/override.conf <<EOF
|
||||
[Service]
|
||||
# Empty line is required
|
||||
ExecStart=
|
||||
ExecStart=/usr/bin/dockerd -H fd:// -H tcp://127.0.0.1:2376
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart docker
|
||||
|
||||
# Pull example docker image
|
||||
sudo docker pull openhpi/co_execenv_python:3.8
|
||||
|
||||
# RVM
|
||||
curl -sSL https://rvm.io/mpapis.asc | gpg --import -
|
||||
curl -sSL https://rvm.io/pkuczynski.asc | gpg --import -
|
||||
curl -sSL https://get.rvm.io | bash -s stable
|
||||
source "/home/vagrant/.profile"
|
||||
|
||||
# ruby
|
||||
rvm install $ruby_version
|
||||
rvm use $ruby_version --default
|
||||
|
||||
# bundler
|
||||
gem install bundler
|
||||
|
||||
######## CODEOCEAN INSTALL ##########
|
||||
cd /home/vagrant/codeocean
|
||||
|
||||
# config
|
||||
for f in action_mailer.yml database.yml secrets.yml code_ocean.yml docker.yml.erb mnemosyne.yml
|
||||
do
|
||||
if [ ! -f config/$f ]
|
||||
then
|
||||
cp config/$f.example config/$f
|
||||
fi
|
||||
done
|
||||
|
||||
# install dependencies
|
||||
bundle install
|
||||
yarn install
|
||||
|
||||
# create database
|
||||
export RAILS_ENV=development
|
||||
rake db:create
|
||||
rake db:schema:load
|
||||
rake db:migrate
|
||||
rake db:seed
|
||||
|
||||
# Always set language to English
|
||||
sudo locale-gen en_US en_US.UTF-8
|
||||
|
||||
#### DOCKERCONTAINERPOOL INSTALL ####
|
||||
../dockercontainerpool/provision.sh
|
||||
Reference in New Issue
Block a user