Improve, restructure documentation and add screenshot

This commit is contained in:
Sebastian Serth
2020-10-20 12:13:30 +02:00
parent 92be9d56ee
commit d25dd41ccf
18 changed files with 3 additions and 1 deletions

167
docs/LOCAL_SETUP.md Normal file
View File

@ -0,0 +1,167 @@
# Local Setup
If available, we prefer a native setup for best performance and less technical issues. Please see below for some details.
## Vagrant
## Install prerequisites
Install Vagrant - https://www.vagrantup.com/docs/installation/
Install VirtualBox - https://www.virtualbox.org/wiki/Downloads
## Install and setup
### Clone repository
Create a local codeOceanRoot: mkdir /path/to/CodeOcean ==> codeOceanRoot = /path/to/CodeOcean
Clone Repository (https://github.com/openHPI/codeocean) to codeOceanRoot
cd codeOceanRoot
### Get Vagrant base image
vagrant box add ubuntu/trusty64
vagrant up
### Trouble shooting
(sometimes, particularly if VirtualBox is running under Windows as the host sysstem, parts of the provision script are) not executed.
vagrant up does not show error messages but later on the trouble starts.
ln -s /etc/nginx/sites-available/code_ocean /etc/nginx/sites-enabled <= Failed (no such directory)
#### Make docker daemon useable without sudo
Infos taken from: http://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo
vagrant ssh
sudo groupadd docker
sudo gpasswd -a ${USER} docker
sudo service docker restart
newgrp docker
apt-get install nginx
ln -s /etc/nginx/sites-available/code_ocean /etc/nginx/sites-enabled
#### If ruby version needs to be updated (as provision.sh is not up-to-date :( )
Infos taken from: http://stackoverflow.com/questions/26242712/installing-rvm-getting-error-there-was-an-error23
vagrant ssh
rvm group add rvm "$USER"
logout and login again
rvm fix-permissions (not necessarily required)
rvm install (requested ruby version)
cd /vagrant
gem install bundler
bundle install
#### Pending migrations
vagrant ssh
cd /vagrant
rake db:migrate
#### Missing config files or anything else goes wrong
Check the according parts of the provision.sh file and try to re-run them directly in the vagrant VM.
All problems that have occurred resulted from a more restrictive rights management in the VMs that run under a Windows host system.
### Start server
vagrant ssh
cd /vagrant
rails s -p 3000 -b 0.0.0.0
### Login to CodeOcean
192.168.59.104:3000
admin@example.org:admin
## Native setup (for macOS)
- Clone this repository:
```shell script
git clone git@github.com:openHPI/codeocean.git
```
- Install PostgreSQL, start it and create a generic postgres user:
```shell script
brew install postgresql
brew services start postgresql
createuser -s -r postgres
```
- Install [NVM](https://github.com/creationix/nvm) and node:
```shell script
brew install nvm
mkdir ~/.nvm
nvm install --lts
```
- Add the following lines to your profile. (e.g., `~/.zshrc`):
```shell script
# NVM
export NVM_DIR=~/.nvm
source $(brew --prefix nvm)/nvm.sh
```
- Install yarn:
```shell script
brew install yarn --ignore-dependencies
```
- Install docker:
```shell script
brew install docker
open /Applications/Docker.app/
```
- Install nginx and adopt its config to forward requests to the **RAW** docker UNIX socket (see [this issue](https://github.com/docker/for-mac/issues/1662) for more details). Only required for macOS!
```shell script
brew install nginx
```
Edit `/usr/local/etc/nginx/nginx.conf`:
1. Change the default port `8080` to `2376` (around line 36).
2. Replace the `location /` with the following and (!) replace `<yourname>` with the output of `whoami`:
```editorconfig
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://unix:/Usrers/<yourname>/Library/Containers/com.docker.docker/Data/docker.raw.sock;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
```
Now, start nginx:
```shell script
brew services start nginx
```
- Install RVM and bundler:
```shell script
gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
curl -sSL https://get.rvm.io | bash -s stable --rails
gem install bundler
```
- Install geckodriver and Firefox for Selenium:
```shell script
brew install geckodriver
brew cask install firefox
```
- Get a local copy of the config files and verify the settings:
```shell script
for f in action_mailer.yml database.yml secrets.yml code_ocean.yml docker.yml.erb mnemosyne.yml
do
if [ ! -f config/$f ]
then
cp config/$f.example config/$f
fi
done
```
- Install gems and yarn files:
```shell script
bundle install
yarn install
```
- Setup your database:
```shell script
rake db:create
rake db:schema:load
rake db:migrate
rake db:seed
```
- Start the server:
```shell script
rails s
```

View File

@ -0,0 +1,11 @@
In order to make containers accessible for codeocean, they need to be reachable via tcp.
For this, the docker daemon has to be started with the following options:
DOCKER_OPTS='-H tcp://127.0.0.1:4243 -H unix:///var/run/docker.sock --iptables=false'
This binds the daemon to the specified socket (for access via the command line on the machine) as well as the specified tcp url.
Either pass these options to the starting call, or specify them in the docker config file.
In Ubuntu, this file is located under: /ect/default/docker
In Debian, please refer to the RHEL and CentOS part under that link: https://docs.docker.com/engine/admin/#/configuring-docker-1

View File

@ -0,0 +1,69 @@
# update apt-get
echo "Update apt-get..."
sudo apt-get update
# upgrade all packages
echo "Upgrade packages..."
sudo apt-get upgrade
#install postgres
if [ ! -f /etc/apt/sources.list.d/pgdg.list ]
then
echo "Add Postgres sources..."
cd /etc/apt/sources.list.d
sudo touch pgdg.list
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ jessie-pgdg main" > pgdg.list'
sudo wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
echo "Done"
else
echo "Postgres sources already added"
fi
sudo apt-get -y --force-yes install postgresql-9.5
# drop postgres access control
if [ -f /etc/postgresql/9.5/main/pg_hba.conf ]
then
if ! sudo -u postgres grep -q CodeOcean /etc/postgresql/9.5/main/pg_hba.conf
then
echo "Drop Postgres access control..."
sudo -u postgres sh -c 'cat >/etc/postgresql/9.5/main/pg_hba.conf <<EOF
#CodeOcean: drop access control
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
EOF'
echo "Done"
echo "Restart Postgres..."
echo sudo service postgresql restart
echo "Done"
else
echo "Postgres access control already dropped"
fi
else
echo "Postgres installation failed"
fi
# create development database
# TODO: extract databasename to variable
if ! (sudo -u postgres psql -l | grep -q codeocean-development)
then
echo "Create database codeocean-development..."
sudo -u postgres createdb codeocean-development || true
sudo -u postgres psql -d codeocean-development -U postgres -c "CREATE USER root;"
sudo -u postgres psql -d codeocean-development -U postgres -c 'GRANT ALL PRIVILEGES ON DATABASE "codeocean-development" to root';
sudo -u postgres psql -d codeocean-development -U postgres -c "CREATE USER debian;"
sudo -u postgres psql -d codeocean-development -U postgres -c 'GRANT ALL PRIVILEGES ON DATABASE "codeocean-development" to debian';
sudo -u postgres psql -d codeocean-development -U postgres -c "CREATE USER codeocean;"
sudo -u postgres psql -d codeocean-development -U postgres -c 'GRANT ALL PRIVILEGES ON DATABASE "codeocean-development" to codeocean';
sudo -u postgres psql -d codeocean-development -U postgres -c 'ALTER DATABASE "codeocean-development" OWNER TO codeocean';
sudo -u postgres psql -d codeocean-development -U postgres -c 'ALTER USER "codeocean" CREATEDB';
echo "Done"
else
echo "Database codeocean-development already exists"
fi
# TODO: create test database

View File

@ -0,0 +1,86 @@
#install docker
if [ ! -f /etc/apt/sources.list.d/backports.list ]
then
#get sources for dependencies
echo "Get apt-get sources for some docker dependencies..."
cd /etc/apt/sources.list.d
sudo touch backports.list
sudo sh -c 'echo "deb http://http.debian.net/debian jessie-backports main" > backports.list'
sudo apt-get update
echo "Done"
#just in case there is some old stuff
echo "Remove legacy stuff...Just in case..."
sudo apt-get purge "lxc-docker*"
sudo apt-get purge "docker.io*"
sudo apt-get update
#install docker dependencies
echo "Install dependencies..."
sudo apt-get install -y --force-yes apt-transport-https ca-certificates gnupg2
echo "Done"
else
echo "Docker dependencies already added."
fi
if [ ! -f /etc/apt/sources.list.d/docker.list ]
then
# get docker sources
echo "Add apt-get sources for Docker..."
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
cd /etc/apt/sources.list.d
sudo touch docker.list
sudo sh -c 'echo "deb https://apt.dockerproject.org/repo debian-jessie main" > docker.list'
sudo apt-cache policy docker-engine
sudo apt-get update
echo "Done"
else
echo "Docker apt-get sources already added."
fi
if [ ! -f /etc/systemd/system/docker.service.d/docker.conf ]
then
echo "Install Docker Engine..."
sudo apt-get install -y --force-yes docker-engine
echo "Done"
echo "Start Docker..."
sudo service docker start
echo "Done"
echo "Run Hello World..."
sudo docker run hello-world
echo "Done"
#set some docker options
echo "Configure Docker..."
sudo mkdir /etc/systemd/system/docker.service.d
cd /etc/systemd/system/docker.service.d
sudo touch docker.conf
sudo sh -c 'cat >>/etc/systemd/system/docker.service.d/docker.conf <<EOF
# code_ocean: enable TCP
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// -D -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock"
EOF'
sudo systemctl daemon-reload
sudo service docker restart
# enable to run docker without sudo
sudo gpasswd -a ${USER} docker
newgrp docker
sudo service docker restart
echo "Done"
else
echo "Docker already installed"
fi
if ! (docker images | grep -q co_execenv_python)
then
echo "Pull Docker images..."
# get docker images
docker pull openhpi/co_execenv_python
docker pull openhpi/co_execenv_java
docker pull openhpi/co_execenv_java_antlr
echo "Done"
else
echo "Docker images already pulled"
fi

View File

@ -0,0 +1,67 @@
if [ ! -f /etc/apt/sources.list.d/nonfree.list ]
then
echo "Get additional sources for apt-get"
cd /etc/apt/sources.list.d
sudo touch nonfree.list
sudo sh -c 'echo "deb http://http.debian.net/debian jessie main non-free contrib" > nonfree.list'
sudo sh -c 'echo "deb-src http://http.debian.net/debian jessie main non-free contrib" >> nonfree.list'
sudo sh -c 'echo "deb http://http.debian.net/debian jessie-updates main contrib non-free" >> nonfree.list'
sudo sh -c 'echo "deb-src http://http.debian.net/debian jessie-updates main contrib non-free" >> nonfree.list'
sudo apt-get update
else
# install utilities
echo "Additional apt-get sources already added"
fi
# install utilities
echo "Install some utils..."
sudo apt-get install -y --force-yes screen
sudo apt-get install -y --force-yes htop
echo "Done"
# install dependencies
echo "Install some libraries..."
sudo apt-get install -y --force-yes git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev
sudo apt-get install -y --force-yes libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev
sudo apt-get install -y --force-yes python-software-properties libffi-dev
sudo apt-get install -y --force-yes libgdbm-dev libncurses5-dev automake libtool bison libffi-dev
sudo apt-get install -y --force-yes libpq-dev
echo "Done"
# get the clock in sync
echo "Install clock synchronization..."
sudo apt-get install -y --force-yes ntp ntpdate
echo "Done"
echo "Install NodeJS..."
# install nodejs
sudo apt-get install -y --force-yes nodejs
echo "Done"
if ! (ruby -v | grep -q 2.3.3)
then
# install rvm
echo "Install RVM..."
gpg2 --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby
source /home/debian/.rvm/scripts/rvm
echo "Done"
# install ruby
echo "Install Ruby 2.3.3..."
rvm install 2.3.3
rvm use 2.3.3 --default
ruby -v
exec bash
echo "Done"
else
echo "RVM and Ruby are already installed"
fi
# install guest additions - required for sharing a folder
echo "Install prerequisites for guest additions..."
sudo apt-get install -y --force-yes dkms build-essential linux-headers-amd64
echo "Done"
echo "Please follow the instructions:"
echo "Insert Guest Additions CD image. VM: Devices=>Insert Guest Additions CD image"
echo "Install Guest Additions"

View File

@ -0,0 +1,10 @@
# Mount Guest Additions and run the installer
echo "Mount Guest Additions and run the installer..."
sudo mount /dev/sr0 /media/cdrom
cd /media/cdrom
sudo sh ./VBoxLinuxAdditions.run
echo "Done"
echo "Please follow the instructions:"
echo "Create Shared Folder. VM: Devices=>VM: Devices=>Shared Folders=>Shared Folders Settings"
echo "Name: codeocean, Path: path to your local codeocaen repository on the host machine."

View File

@ -0,0 +1,7 @@
echo "Mount Shared Folder..."
mkdir /home/debian/codeocean_host
sudo mount -t vboxsf -o rw,uid=1000,gid=1000 codeocean /home/debian/codeocean_host
# Enable automount during startup
sudo sh -c 'echo "sudo mount -t vboxsf -o rw,uid=1000,gid=1000 codeocean /home/debian/codeocean_host" >> /home/debian/.bashrc '
echo "Done"

View File

@ -0,0 +1,28 @@
############# codeocean install ###########################
cd /home/debian/codeocean_host
#install rails and bundler
echo "Install Rails..."
gem install rails
echo "Done"
echo "Install Bundler..."
gem install bundler
echo "Done"
# install required gems
bundle install
# copy config files
for f in action_mailer.yml database.yml secrets.yml sendmail.yml smtp.yml code_ocean.yml
do
if [ ! -f config/$f ]
then
cp config/$f.example config/$f
fi
done
# Manual Task:
# if necessary adjust db config
echo "Check if settings in database.yml correspond with your database setup."
cat /home/debian/codeocean_host/config/database.yml

View File

@ -0,0 +1,8 @@
# create, migrate, and seed database tables
cd /home/debian/codeocean_host
export RAILS_ENV=development
echo "load, seed, migrate"
rake db:schema:load
rake db:seed
rake db:migrate

View File

@ -0,0 +1,165 @@
# Prerequisites:
# 1 Download Debian iso image. http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-cd/debian-8.6.0-amd64-netinst.iso
# 2 Create Debian VM in VirtualBox:
# - without GUI
# - without webserver (we do not want an apache2 but an nginx server)
# - with ssh ()
# 2 Create 2 users
# - debian/debian
# - root/root
# Manual preparation:
# Login as root
su
# install sudo
apt-get install -y sudo
# add user debian to sudoers and enable this user to sudo without password (do not do this on a production machine)
# or change the line after finishing the installation
cd /etc/sudoers.d
touch debian
echo "debian ALL=(ALL) NOPASSWD:ALL" >> debian
# echo "debian ALL=(ALL:ALL) ALL" >> debian # production systems
# return to no-root user again
exit
# Running the following directly on the VM command line is inconvenient
# Therefore enable login via ssh from Host
# The best way to login to a guest Linux VirtualBox VM is port forwarding.
# By default, you should have one interface already which is using NAT.
# Then go to the Network settings and click the Port Forwarding button. Add a new Rule:
# Protocol TCP Host port 3022, guest port 22, name ssh, other left blank.
# That's all! Please be sure you don't forget to install an SSH server:
# To SSH into the guest VM, write:
# ssh -p 3022 user@127.0.0.1
# http://stackoverflow.com/questions/5906441/how-to-ssh-to-a-virtualbox-guest-externally-through-a-host
#=======================================================================================================
# Install postgres
# run script:
debian_installer/setup_debian_1_install_postgres.sh
# Install docker
# run script:
debian_installer/setup_debian_2_install_docker.sh
# Install dependencies, utils, rvm, ruby, node
# run script:
debian_installer/setup_debian_3_install_depencies_and_utils.sh
##################################local installation on VirtualBox only##################
# Before running the next script, the Guest Additions CD image needs to be inserted via VBox GUI
# Devices=>Insert Guest Additions CD image"
# When that is done run the next script
debian_installer/setup_debian_4_install_guest_additions.sh
# Before running the next script, a Shared Folder has to be created via VBox GUI
# Devices=>Shared Folders=>Shared Folders Settings
# Folder Name: codeocean, Folder Path: path to your local codeocean repository on the host machine.
# Automount, Make Permanent
# When that is done run the next script
debian_installer/setup_debian_5_mount_shared_folder.sh
##################################local installation on VirtualBox only##################
# Install rails and bundler
# run script:
debian_installer/setup_debian_6_setup_codeocean.sh
# Create, seed, and migrate database tables
# run script:
debian_installer/setup_debian_7_create_tables.sh
# Add Port Forwarding for Rails server:
# Protocol TCP Host port 3030, guest port 3000, name CodeOcean, other left blank.
# That's all!
# Start Puma server on VM (since we upgraded to rails 4.2.5, it is necessary to specify the address here as well. Otherwise, we can't connect from the host machine)
# rails s -b 0.0.0.0 -p 8080
# To connect to Ruby app use
#http://127.0.0.1:3030
#The following is required so that CodeOcean can connect back to openHPI local
# Setup a second networking interface
# 1. Host-only vboxnet0 (ip-address: 192.168.59.104)
# 2. NAT with all the portforwarding stuff as described above
# Edit /etc/network/interfaces in Guest machine:
# 1. check for available interfaces:
# ls /sys/class/net ===> docker0 eth0 eth1 lo
# 2. edit network configuration:
# sudoedit /etc/network/interfaces
# and add the following lines:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
# allow-hotplug eth0
# iface eth0 inet dhcp
#Host-only interface
auto eth0
iface eth0 inet static
address 192.168.59.104
netmask 255.255.255.0
network 192.168.59.0
broadcast 192.168.59.255
#NAT interface
auto eth1
iface eth1 inet dhcp
# See also:
# http://askubuntu.com/questions/293816/in-virtualbox-how-do-i-set-up-host-only-virtual-machines-that-can-access-the-in
# !!!!!Attention!!!!!!!!
# Start openHPI Local as:
# http://{host.ip}:3000/
# e.g. http://192.168.178.33:3000/
# set LTI Provider in course as:
# http://192.168.59.104:3030/lti/launch
# Access VBox with static IP and port-forwarding
# SSH:
# ssh -p 3022 debian@192.168.59.104
# CodeOcean:
# http://192.168.59.104:3030
#TODO production:
# require passwd for sudo again.
# cd /etc/sudoers.d
# echo "debian ALL=(ALL:ALL) ALL" > debian
#TODO production: Install nginx
# install nginx
# echo "Install NGINX..."
# sudo apt-get install -y --force-yes nginx
# echo "Done"

BIN
docs/implement.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 870 KiB

12
docs/webpython/Dockerfile Normal file
View File

@ -0,0 +1,12 @@
FROM ubuntu:14.04
MAINTAINER "Martin v. Löwis"
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV PYTHONPATH $PYTHONPATH:/usr/lib/python3.4:/workspace
ENV PATH $PATH:/usr/lib/python3.4
ADD assess.py /usr/lib/python3.4/assess.py
ADD webpython.py /usr/lib/python3.4/webpython.py
RUN rm /usr/lib/python3.4/turtle.py
ADD turtle.py /usr/lib/python3.4/turtle.py
RUN adduser --disabled-password --gecos Python python
USER python

58
docs/webpython/Makefile Normal file
View File

@ -0,0 +1,58 @@
CODEMIRROR=html/js/codemirror.js
all: logs
logs:
mkdir logs
# These are put into source control
generated: $(CODEMIRROR)
$(CODEMIRROR): CodeMirror/lib/codemirror.js CodeMirror/mode/python/python.js
(cd CodeMirror; bin/compress codemirror python) > $@
run: all kill
twistd -l logs/webpython.log -y webpython.tac
runpy: killpy
twistd --pidfile pylaunch.pid -l logs/pylaunch.log -y pylaunch.tac
manager: all killmanager
twistd --pidfile manager.pid -l logs/manager.log -y manager.tac
kill:
if [ -f twistd.pid ];\
then\
kill `cat twistd.pid`;\
fi
killpy:
if [ -f pylaunch.pid ];\
then\
kill `cat pylaunch.pid`;\
fi
killmanager:
if [ -f manager.pid ];\
then\
kill `cat manager.pid`;\
fi
docker:
docker.io build --rm -t webpython .
update:
git pull
make docker
service webpython-worker restart
rmexited:
docker.io ps -a|grep 'Exit '|awk '{print $$1;}'|xargs docker.io rm
rmi:
docker.io rmi $$(docker.io images | grep "^<none>" | awk '{print $$3;}')
killold:
-docker.io ps | egrep 'hours? ago' |awk '{print $$1}'|xargs docker.io kill
-killall -o 30m -9 python3

47
docs/webpython/README.md Normal file
View File

@ -0,0 +1,47 @@
Local setup
===========
1. `git checkout webpython-hybrid`
2. Make sure to install all dependencies and migrations:
rake db:migrate
bundle install
3. Create a new docker image containing the Turtle library and the i/o wrapper:
cd webpython
docker build -t IMAGE_NAME .
4. Configure your Docker host at `config/docker.yml.erb`. Make sure to add a websocket host, for example like this (this is probably different for you):
host: tcp://localhost:2375
ws_host: ws://localhost:2375
5. Run the CodeOcean server with `rails s -p 3333`
6. Login with admin@example.org (pw: admin) and create a new execution environment picking the newly created Docker image from the dropdown. Set the initial command to:
cd /usr/lib/python3.4 && python3 webpython.py
7. Create a new exercise for the newly created execution environment with an arbritrary main file.
8. Implement the exercise. The code below can be used as an example to see the canvas and I/O in action:
import turtle
wn = turtle.Screen()
alex = turtle.Turtle()
# i/o test
print("hello!")
print("please enter your name")
name = input()
print("your name is", name)
# canvas test
alex.forward(50)
alex.right(90)
alex.forward(30)
alex.right(90)
alex.forward(30)
wn.mainloop()

1
docs/webpython/assess.py Normal file
View File

@ -0,0 +1 @@
# moved to dockerfiles/ubuntu-python

1
docs/webpython/turtle.py Normal file
View File

@ -0,0 +1 @@
# moved to dockerfiles/ubuntu-python/turtle.py

View File

@ -0,0 +1 @@
# moved to dockerfiles/ubuntu-python/webpython.py