Conducting research in the areas of data engineering, data management, and machine learning systems requires the ability to deal with scientific literature in these areas as well as to design, implement, and evaluate prototypes. To facilitate these skills, the DAMS Lab group (FG Big Data Engineering) at TU Berlin offers a seminar and a project on Large-scale Data Engineering as a combined module (12 ECTS), which can be taken by bachelor and master students. Taking both seminar and project is the ideal preparation for a bachelor/master thesis with our group. Alternatively, only bachelor students may take the seminar as a separate module (3 ECTS) and the project as a separate module (9 ECTS).
Modules and assigned degree programsAt the beginning of the semester, students hear introductory lectures on reading scientific papers, finding related work, writing high-quality scientific papers, and giving a high-quality scientific presentation. Each student is assigned an initial paper to read and understand. After that, students search for related work and write a short summary of the assigned paper, including an overview of related work. At the end of the semester, each student gives a slide presentation in front of the group, followed by a discussion of the topic.
This semester's umbrella topic: Robust and Adaptive Query Processing
Traditionally, database query processing is divided into an optimization phase, which determines an optimal plan for the query, and an execution phase, which executes this plan. During optimization, different logically equivalent plans are enumerated and the plan with the lowest cost with respect to some cost model is chosen. Cost estimation is largely based on estimates of the cardinalities of intermediate results. Unfortunately, these estimates are often quite wrong resulting in bad query execution plans that may take orders of magnitude longer to execute than the optimal plan. Moreover, additional unknowns further complicate the efficient query processing, e.g., unknown properties of base data and input datasets, the access to external data sources, query parameters, and the system utilization at run-time. This semester, we deal with a broad range of research papers that address these challengens through (a) improved creation/management of statistics, (b) robust query optimization, and (c) adaptive query processing.
List of topics: (tba)
Submission & deadlinesAt the beginning of the semester, students pick a project from a provided list, devise an initial design and implement an initial prototype including tests and documentation. The students present the initial prototype in front of the course in the middle of the semester. Furthermore, they conduct extensive experiments to prove the quality and properties of their prototype. The results of these experiments guide the further development of a final prototype, which the students present in front of the course at the end of the semester. The projects are augmented by regular discussion rounds with a project mentor throughout the semester.
The topics of the project are independent of the seminar. We will offer tasks in a wide range of components of data management and machine learning systems. Each individual project will be conducted in the context of one of the two systems developed by our group (and other collaborators) as part of our research:
Thereby, students get the chance to make meaningful contributions to free open-source projects. The projects can be done either individually or in teams of up to three students (with the expected amount of work proportional to the team size).
List of topics (tba)
Submission & deadlines