masterElmar/flatscraper
1
0
Fork 0
mirror of https://gitlab.dit.htwk-leipzig.de/fsr-im/tools/flatscraper.git synced 2026-01-15 23:32:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
27 Commits 1 Branch 0 Tags
main
Commit Graph

27 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Elmar Kresse
825ffe7743 Merge branch '5-automate-lwb-scraping' into 'main' 
Resolve "automate lwb scraping"

Closes #5

See merge request fsr-im/tools/flatscraper!2
 2025-08-15 16:01:20 +02:00
Elmar Kresse
401eafcca5 style: clean up whitespace and formatting in main.py and webhook.py 2025-08-15 15:17:35 +02:00
Elmar Kresse
2c8cea7645 feat: automate session cookie retrieval and remove manual session ID setup 2025-08-15 15:06:52 +02:00
Elmar Kresse
78152dafc0 added playwright lwb scraping with chromium headless 2025-08-15 15:00:29 +02:00
Elmar Kresse
4f62e2709a Merge branch 'fix-wogetra' into 'main' 
Fixed: Extract specific details (rooms, ...) in wogetra which caused a error if details is empty

See merge request fsr-im/tools/flatscraper!1
 2025-05-28 12:01:57 +02:00
Max Löbel
6094d56a08 Fixed: Extract specific details (rooms, ...) in wogetra which caused a error if details is empty 2025-05-27 22:55:34 +02:00
Elmar Kresse
9933141178 docs: remove redundant introduction from README and streamline content 2025-04-02 11:57:34 +02:00
Elmar Kresse
c852f9e305 docs: enhance README with improved formatting, features, installation instructions, and emojis 2025-04-02 11:45:16 +02:00
Elmar Kresse
683403e8bf feat: update property handling in VLW scraper and main process, fix image URL formatting 2025-02-16 22:58:19 +01:00
Elmar Kresse
9bf5155e9f feat: add VLW owner handling and image scraping logic in LWB and VLW scrapers 2025-02-16 22:40:35 +01:00
Elmar Kresse
395434da1c feat: add VLW property scraping functionality to main process 2025-02-16 19:20:27 +01:00
Elmar Kresse
92bb5b2e85 feat: update Docker setup, add environment variables, and enhance property scraping logic 2025-02-10 10:02:53 +01:00
Elmar Kresse
f16116040d refactor: clean up unused code and comments in BGL scraper 2025-02-09 19:46:50 +01:00
Elmar Kresse
17078d10f6 feat: enable property scraping from Wogetra, LWB, and Lipsia in main scraping process 2025-02-09 19:41:23 +01:00
Elmar Kresse
cd552507bd chore: update .gitignore to exclude __pycache__ directories 2025-02-09 19:35:42 +01:00
Elmar Kresse
9273435e7f chore: remove compiled Python bytecode files from __pycache__ directories 2025-02-09 19:35:17 +01:00
Elmar Kresse
f1c9816d9e feat: add BGL scraper and integrate into main scraping process; update image scraping logic 2025-02-09 19:33:53 +01:00
Elmar Kresse
010a3249cf fix: update known_properties.json and modify scraper.py with new session credentials 2025-02-06 23:19:33 +01:00
Elmar Kresse
a6c147a18d feat: enhance scraper with session management and query parameter setup 2025-02-05 01:17:33 +01:00
Elmar Kresse
9be18e4d06 fix: correct longitude variable name in Lipsia scraping function 2025-02-04 23:25:37 +01:00
Elmar Kresse
7effa15ddc refactor known_properties.json and update scraper logic for improved session handling and image scraping 2025-01-28 10:50:32 +01:00
Elmar Kresse
b468fc6578 update known_properties.json and modify scraper.py to enhance session handling 2025-01-14 11:24:37 +01:00
Elmar Kresse
b2ef4eb5f2 add scraping functionality for Lipsia and improve error handling for image downloads 2025-01-13 11:57:20 +01:00
Elmar Kresse
1093f49438 renamed file gitlab ci 2025-01-13 10:58:00 +01:00
Elmar Kresse
c3142e3378 add Dockerfile and CI configuration for building and pushing Docker image 2025-01-13 10:56:42 +01:00
Elmar Kresse
b337b7c2f8 add initial scraping functionality and related utilities 2025-01-13 10:44:01 +01:00
Elmar Kresse
9a0a72f640 add README.md with project description 2025-01-11 12:29:48 +01:00