This lab uses your class-exercises-fall2026 fork (not lab03-exercises).
On GitHub: open your fork of class-exercises-fall2026 and sync it with the class repo (csci338/class-exercises-fall2026) using Sync fork / Update branch.
On your computer, in your local class-exercises-fall2026 folder:
Confirm it appears in pyproject.toml and is installed:
poetry show
2.3 Run code in the virtual environment
Create lab05-experiments.py inside poetry-demo with this starter code:
import requests
defmain():
print("hello world")
# user_agent makes it seem like the request is coming from a web browser (versus a bot)
user_agent = {'User-agent': 'Googlebot/2.1'}
response = requests.get("https://new.cs.unca.edu/", headers=user_agent)
print(response.content)
if __name__ == "__main__":
main()
Run it inside Poetry’s environment:
poetry run python lab05-experiments.py
You should see HTML from the CS homepage.
Run the same file outside the environment:
python3 lab05-experiments.py
You should get an error because requests is not installed for system Python.
Use BeautifulSoup helpers to collect links. You may ask Google or ChatGPT: “How do I extract URLs using Beautiful Soup?”
Optional readiness extension (Practice)
After the basic crawler works, choose one:
Deduplicate the links while preserving their original order.
Count how many links point to each domain using a Python dictionary.
Filter the output to include only links whose domain ends in unca.edu.
Write at least two small tests for the transformation logic using a hand-written list of URLs; do not make your tests depend on the live website. In a comment, name the Java collection you would use for the same algorithm and explain why.
2.5 Remove and re-add dependencies
Remove the packages:
poetry remove requests
poetry remove bs4
Run again and notice what happens:
poetry run python lab05-experiments.py
Add them back:
poetry add requests
poetry add bs4
Run again and notice what happens:
poetry run python lab05-experiments.py
Before moving on
poetry run python lab05-experiments.py prints the CS homepage links
I saw the error outside the Poetry environment (system python3)
I removed and re-added requests / bs4 and observed what happened
3. npm (Node.js)
npm is the default package manager for Node.js.
3.1 Install Node.js (if needed)
Check your version:
node -v
If Node is missing, install it (Mac or WSL steps below).