Big Data Engineering

Dear students,

we have just uploaded the project under Materials -> Project. A detailed project description including instructions for the submission can be found in the Project Description and the project.pdf file in the root directory of the project (same file). The submission deadline for the project is Sunday, 29.06.25, 23:59! If you correctly joined a team, only one team member has to upload your submission. Please make sure that every team member is able to see the uploaded submission on their personal status page. Only then, all team members will receive the points.

Additionally, each group must participate in a code review after the project submission. We offer seven code review slots:
   •  Slot 1: Wednesday 02.07.25, 10:00 - 12:00
  •   Slot 2: Wednesday 02.07.25, 12:00 - 14:00
  •   Slot 3: Wednesday 02.07.25, 14:00 - 16:00
  •   Slot 4: Wednesday 02.07.25, 16:00 - 18:00
  •   Slot 5: Thursday     03.07.25, 12:00 - 14:00
  •   Slot 6: Thursday     03.07.25, 14:00 - 16:00
  •   Slot 7: Thursday     03.07.25, 16:00 - 18:00

More detailed information about the code reviews will be published at the end of the project, but keep in mind that every team members must know and understand all code. Team members who are not able to explain parts of the code do not get points for the project.

Please register for the code review slot on your personal status page by Sunday, 29.06.25, 23:59! Note that only one team member is allowed to register for the entire team, so please coordinate internally! Registration is on a first-come-first-served basis.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 06. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we would like to remind you that the deadline for creating or joining a team for the project submission is tomorrow at 23:59. This registration is mandatory to participate in the project! 

If you are still looking for a team, you can use the forum. If you are not able to find a team in the forum or somehow else, please still create a team in the CMS for yourself. We will merge teams with less than three people after the deadline and inform you about your team members.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 07. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 05. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

the programming project will start next week. We allow submitting the project in teams of 3 to 4 students. Please note, that submissions of smaller groups will not be graded and thus the exam admission cannot be achieved.

To register your team, you have to use the Team Groupings functionality of the CMS. This means for you that a team member creates a group in the CMS on the personal status page and invites all other team members to the group. You can create your team until Wednesday 11.06.25 at 23:59, after that it is not possible to change the team.

To find a team, you can use our forum.

Kind regards,
Your BDE Team

Dear students,

since the number of participants in the tutorials is continuously decreasing, we have decided to merge tutorials that take place at the same time from now on. Here is an overview of the tutorial slots starting this week.

    •    Tutorial 1 & 2: Wednesday, 10:15 in E1 1, 3.06
    •    Tutorial 3 & 4: Wednesday, 12:15 in E1 1, 3.06
    •    Tutorial 5 & 6: Wednesday, 14:15 in E1 1, 3.06
    •    Tutorial 7 & 8: Wednesday, 16:15 in E1 1, 3.06

You can find a concise overview in our Timetable as well.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 06. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 05. The solutions will be published on Tuesday next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 04. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 04. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 03. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 03. The solutions will be published after the mini-test next week.

Note that there is again a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 02. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

we have just uploaded Assignment 02. The solutions will be published after the mini-test next week.

Note that there is a practical exercise using a Jupyter notebook. We won't publish solutions for that exercise but you are welcome to ask and discuss in your tutorial.

Kind regards,
Your BDE Team

Dear students,

we have just published the results of Mini-Test 01. You can view your results on your personal status page.

You may inspect your mini-test at the end of your regular tutorial tomorrow (if you were able to state the correct tutorial number on your mini-test). If you do not have a tutorial assigned (or were not able to state the correct tutorial number on your mini-test), you are welcome to inspect your mini-test during the office hour next week.

Remember to bring your student ID for inspection.

Kind regards,
Your BDE Team

Dear students,

the mini-test slot registration is now closed. The first mini-test will take place tomorrow, 12:15 - 12:45 in E2 2 - lecture hall 0.01 (Günter-Hotz). More information can be found in the Timetable.

General Information and Procedure

• There are two mini-test slots. The first slot will start at 12:15, the second slot will start once the first one has finished, i.e., approximately at 12:30. Both slots will last 15 minutes.
• You are only allowed to participate in the slot you have registered.
• Be in front of the lecture hall at least 5 minutes early. Please wait until you are instructed to enter the lecture hall. Note that we will ask you to enter the lecture hall through the lower doors and leave through the upper doors to avoid communication between the two slots.
• Use a document-safe, permanent pen such as a ballpoint pen. No additional materials are permitted. If necessary, we provide additional paper.
• As the mini-tests do not have a cover page, we will distribute the mini-tests to the individual seats once you are seated. Please leave every second seat and every third row empty.
• Please fill in your name, your matriculation number, and your tutorial number on the first page. We need your tutorial number to make sure you are able to inspect your mini-test in your tutorial.
• You can start working on the mini-test as soon as you receive a copy and continue until we collect your copy again. We will hand out and collect the mini-tests in the same order so that everyone gets the same amount of processing time. Continued writing after we have asked you to stop to collect your mini-test will be considered as attempted cheating.
• Take your student ID card with you to your seat.

Kind regards,
Your BDE Team

Dear students,

we have now assigned you to your tutorial groups. You can find your assigned tutorial on your personal status page. The tutorials will start this week. More information can be found in the Timetable.

If you were assigned to Tutorial 9 (Wednesday, 00:00 No Tutorial) but still want to attend a tutorial at some point, you are welcome to attend any other tutorial.

We will upload the tutorial slides before each tutorial and the solutions afterwards.

Kind regards,
Your BDE Team

Dear students,

we would like to remind you to register for a mini-test slot.

Currently, we offer two mini-test slots at the following times:
    •    Slot 1: Tuesday, 12:15 - 12:30
    •    Slot 2: Tuesday, 12:30 - 12:45

Registration is mandatory to attend the mini-tests and works on a first-come-first-served basis. Depending on the number of students, we will add a third slot from 12:45 - 13:00 or in parallel to one of the other two slots, s.t. everyone who wants to attend the mini-tests will be able to register.

More information about the mini-test procedure will follow after the registration has been closed.

Please register for a mini-test slot on your personal status page by Sunday, 20.04.2025, 23:59!

Kind regards,
Your BDE Team

Dear students,

we would like to remind you to set your preferences for the tutorials starting next week.

We offer eight regular tutorials at the following times:
    •    Slot 1: Wednesday, 00:00 No Tutorial: This is a dummy tutorial! Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices.
    •    Slot 2: Wednesday, 10:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E1 1, 2.06
    •    Slot 3: Wednesday, 12:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E2 5, U.36
    •    Slot 4: Wednesday, 14:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E2 5, U.36
    •    Slot 5: Wednesday, 16:15 Tutorial: Two regular tutorials at this time slot in E1 1, 3.06 and E2 5, U.36

In the tutorials, you have the opportunity to briefly discuss the solution of the previous assignment sheet and to inspect your mini-test. However, the main focus will be a recap of the lecture material by providing additional exercises. Please note, that both the solutions of the assignment sheets as well as the tutorial slides including solutions will be timely uploaded to our Materials.

Because of this, experience has shown that some students will not attend the tutorials in person. Therefore, we also created an additional (pseudo) tutorial (Slot 1: Wednesday, 00:00). Please give preference to this tutorial if you do not plan to attend the tutorials in general and rule out the other tutorial choices. With this, we can allow students that plan to attend, to more likely get their preferred choice. Furthermore, we can better assess how many in-person tutorials we really need. Of course, you can always attend a tutorial if you change your mind.

Please submit your tutorial preferences on your personal status page by Sunday, 20.04.2025, 23:59!

Kind regards,
Your BDE Team

Dear students,

in the course of this lecture, we will use Python and in particular Jupyter Notebooks to explain various concepts. For a few notebooks, we also require additional software (e.g. PostgreSQL). Therefore, we provide a Virtual Machine using Docker that has all the required software and systems already installed. You can find detailed instructions at the following Github repository. The link is also available under Information > Docker.

We spent a lot of time preparing the tools and the instructions. Please read them carefully before asking questions.

Please note, that you are also able to run most of the notebooks by just having a Python installation with the required packages. For more information see Information > Manual Installation.

If you still have problems with the installation, you can always ask a question in the forum (BDE25 > Technical). Additionally, there will be a special Docker support office hour on Tuesday, 15.04.2025, 12:15 - 14:00 in E2 2 - Lecture Hall 0.01 (Günter-Hotz).

Kind regards,
Your BDE Team