Conducting research in the areas of data engineering, data management, and machine learning systems requires the ability to deal with scientific literature in these areas as well as to design, implement, and evaluate prototypes. To facilitate these skills, the DAMS Lab group (FG Big Data Engineering) at TU Berlin offers a seminar and a project on Large-scale Data Engineering as a combined module (12 ECTS), which can be taken by bachelor and master students. Taking both seminar and project is the ideal preparation for a bachelor/master thesis with our group. Alternatively, only bachelor students may take the seminar as a separate module (3 ECTS) and the project as a separate module (9 ECTS).
Modules and assigned degree programsIn the beginning of the semester, students hear introductory lectures on reading scientific papers, finding related work, writing high-quality scientific papers, and giving a high-quality scientific presentation. Each student selects a topic, reads and understands the given paper, searches for related work, and writes a short summary of the assigned paper. In the end of the semester, each student gives a slide presentation in front of the class.
This semester's umbrella topic: Robust and Adaptive Query Processing
Traditionally, database query processing is divided into an optimization phase, which determines an optimal plan for the query, and an execution phase, which executes this plan. During optimization, different logically equivalent plans are enumerated and the plan with the lowest cost with respect to some cost model is chosen. Cost estimation is largely based on estimates of the cardinalities of intermediate results. Unfortunately, these estimates are often quite wrong resulting in bad query execution plans that may take orders of magnitude longer to execute than the optimal plan. Moreover, additional unknowns further complicate the efficient query processing, e.g., unknown properties of base data and input datasets, the access to external data sources, query parameters, and the system utilization at run-time. This semester, we deal with a broad range of research papers that address these challengens through (a) improved creation/management of statistics, (b) robust query optimization, and (c) adaptive query processing.
Submission & deadlinesIn the beginning of the semester, students/teams select a project topic from a provided list. Then, they design and implement a high-quality prototype and prove the value of their contribution through extensive tests, experiments, and documentation. The project ends with a presentation and defense of the results in front of the class.
The topics of the project are independent of the seminar. We will offer tasks in a wide range of components of data management and machine learning systems. Each individual project will be conducted in the context of one of the two systems developed by our group (and other collaborators) as part of our research:
Thereby, students get the chance to make meaningful contributions to free open-source projects. The projects can be done either individually or in teams of up to three students (with the expected amount of work proportional to the team size).
List of topics (updated Oct 22)
Submission & deadlines