HTWKalender

Scrape information about Seminar Groups and dates.

Run Development Instance with

docker compose up --build 

Go to Admin UI

➜ Webinterface: http://127.0.0.1/

➜ Admin UI: http://127.0.0.1/_/

➜ API: http://127.0.0.1/api/

For first login use the following credentials:

Email:

demo@htwkalender.de

Password:

htwkalender-demo

Fetch Data from HTWK

Execute the following api calls to fetch data manually from HTWK and store it in the database:

Both api calls need a token to be executed. You can get a token by logging in to the admin ui and copy the token from the local storage. Add this attribute to the request header of the api call:

Authorization : eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDE3MDA3ODQsImlkIjoicnA0Ym54YXNyczM5emR4IiwidHlwZSI6ImFkbWluIn0.j7Bt3-uaZ8CoNt8D9Oxjk7ZwvHDGZJy1xe3aq4BID3w

The first command will fetch all groups and store them in the database. This should be done quick in a few seconds (0-5s). When you execute the command again, it will update the groups in the database and only return new added groups.

http://127.0.0.1/api/fetch/groups

For fetching the plans, you can use the following command. This will fetch all plans for all groups and store the events in the database. It's done for all current existing events (ws/ss). The whole process takes a while (30s-2min), depending on the amount of groups and events. Stay for this time on the page and wait for the response.

http://127.0.0.1/api/fetch/events

After you fetched the events you can optional fetch sport events. This will scrape the sport events from the HTWK website and store them in the database. This will take a few seconds (2s-10s).

http://127.0.0.1/api/fetch/sports

View/Filter/Search in Admin UI

If you want some easy first api endpoints and data views, you can use the Admin UI.

Schedules

In our project we used schedules to fetch data from HTWK and store it in the database. So they get periodically executed and keep the database up to date.

You can find the schedules in the following directory:

service/addSchedule.go

Currently, there are 3 schedules:

• FetchGroupsSchedule
• FetchEventsSchedule
• FetchSportsSchedule