CSCI 598B - Linguistic Human-Robot Interaction

Spring 2018

TR 12:30-1:45

Berthoud Hall 206

1 Course Description

The course explores the capabilities necessary for autonomous robots to participate in natural language dialogue with human partners, including language and gesture understanding, action selection and execution, language and gesture generation, and integrated robot architectures.

2 Instructor: Tom Williams

3 Quick Reference

4 Course Goals

By the end of the semester, students should be able to:

  1. Recognize the fundamental stages of language (and gesture) understanding and generation.
  2. Understand the tradeoffs between algorithms used for language (and gesture) understanding and generation.
  3. Design and implement algorithms for language and gesture understanding and generation.
  4. Design and implement components for integrated robot architectures.
  5. Understand how to read, analyze, and effectively communicate the contents of a scientific paper.
  6. Carry out independent or group research and effectively communicate the results.

5 Course Format

The course will be conducted as a 3-credit-hour seminar, where class time is focused on students discussing scientific papers they have read for homework. Each class, one pair of students will be responsible for leading class discussions on the topic-du-jour, and for presenting a research paper related to that topic. Outside of class, in addition to reading articles, students will have several significant programming projects, and will conclude the course with a final project, experimental evaluation, oral presentation, and final essay.

6 Computer Tools

For this course, you will need Linux on your personal computer. To get it set up, you should either dual boot it (if you feel comfortable doing so), or install a Linux virtual machine. To do so, you will need VM software such as virtualbox (free), and an image for a recent version of Ubuntu.

Within Linux, you will need to install Java 8, the RXTX-Java library, git, and ssh, as well as the ADE implementation of the DIARC architecture, which will be made available to you.

7 Assessment and Grading

Students will be assessed with using the following elements.

  • Paper presentations and discussion leading (15%)
    • For each class session, you will be assigned two scientific articles to read: one mandatory, and one optional. The latter 65 minutes of each class period will be in the hands of two graduate students. One student will be responsible for leading a 30-minute class discussion on the mandatory reading; the other student will be responsible for a 15-minute presentation on the optional article, and for leading a 20-minute class discussion on the optional reading.
  • Discussion participation (15%)
  • Bi-weekly quizzes (and quiz question submission) (10%)
    • The night before each class, you will be asked to submit five quiz questions on the mandatory article. The next day's class will begin with a ten-minute, five-question quiz using questions sampled from all submitted questions.
  • Semi-weekly homework assignments (40%)
    • There will be five programming assignments, each of which will build off the previous assignments and take 2-3 weeks to complete.
  • Final Project (20%)
    • For your final project, you will create a novel algorithm that uses a robot's situated context to handle task-based natural language dialogues in a way that improves on established methods which do not take the robot's context into consideration, implement this algorithm as part of a component of the ADE robotic architecture, and perform an evaluation demonstrating the benefits of your approach stemming from its use of the robot's situated context.
    • You are encouraged to use the NL pipeline you have developed throughout the semester, and to complete your final project by significantly extending / improving one of the components you developed as part of that pipeline, or by adding an additional component which was not covered as homework.
    • Your final project grade will consist of three components: (1) Your final project implementation; (2) A final paper writeup for your project in the style of an academic conference paper; and (3) A final presentation regarding your project.
    • No exams!

8 Course Procedures

8.1 Communications

All course announcements will be made through Canvas, so please check it frequently. We will be using Piazza for class discussion. This system is designed for getting help quickly and easily from classmates and the instructor. Rather than emailing questions to the teaching staff, we encourage you to post your questions on Piazza. *To be clear: all messages to the instructor should be sent via Piazza; there is no guarantee that emails to the instructor will be seen or responded to.* You are also encouraged to help each other, so long as they follow the guidelines listed below:

  • If your question contains any code or gives away any portion of an answer, you must make it private (i.e., only visible to instructors) or we will be forced to report it as a violation of academic integrity.
  • If your question does not contain any code or contain any portion of answer to a homework question, please make it public, so that other students can see your question, and potentially help you answer it.
  • In order to schedule office hours outside the posted times, post a private question on Piazza.
  • Your homework must be entirely your own work. While you may discuss homework verbally with others, all work must be done yourself, and you must not show each other your written homework assignments or code. This is a violation of academic integrity.
  • On your assignments, please acknowledge, in writing, students with whom you discussed problems or course material.

    These policies apply in addition to all departmental / university policies.

8.2 Electronic Devices Policy

Research has demonstrated that the use of electronic devices (e.g., laptops, tablets, cellphones) significantly impairs the learning of students using them. What is more, the learning of students seated near students using electronic devices is impaired. For these reasons, no electronic devices will be allowed in the classroom. An exception may be made in the case of a disability, if the student approaches the instructor beforehand and an arrangement is agreed to. And of course, this does not apply during lab sessions or when otherwise indicated by the instructor.

8.3 Academic Integrity

Academic integrity is taken very seriously. While plagiarism may be the worst violation of academic integrity (and as such we may examine your work for plagiarism using automated heuristics), we are required to report any suspected violation of academic integrity to the University's Judicial Officer. Penalties for violation can be very severe, including suspension or expulsion. If any student does not understand these terms or any outlined in The Academic Code of Conduct it is his/her responsibility to talk to the instructor.

The policies described below will be enforced in addition to all departmental / university policies.

8.3.1 Collaboration Policy for Programming Projects in CS Courses

The following policy exists for all CS courses in the EECS department. This policy is a minimum standard; your instructor may decide to augment this policy.

  • If the project is an individual effort project, you are not allowed to give code you have developed to another student or use code provided by another student. If the project is a group project, you are only allowed to share code with your group members.
  • You are encouraged to discuss programming projects with other students in the class, as long as the following rules are followed:
    • You view another student’s code only for the purpose of offering/receiving debugging assistance.
    • Students can only give advice on what problems to look for; they cannot debug your code for you.
    • All changes to your code must be made by you.
    • Your discussion is subject to the empty hands policy, which means you leave the discussion without any record [electronic, mechanical, or otherwise] of the discussion.
    • Any material from any outside source such as books, projects, and in particular, from the Web, should be properly referenced and should only be used if specifically allowed for the assignment.
    • To prevent unintended sharing, any code stored in a hosted repository (e.g. on GitHub) must be private. For group projects, your team members may, of course, be collaborators.
    • If you are aware of students violating this policy, you are encouraged to inform the professor of the course. Violating this policy will be treated as an academic misconduct for all students involved. See the Student Handbook for details on academic dishonesty.

8.3.2 Collaboration Policy for Homework

The following policy applies to homework assignments other than programming projects.

  • You can discuss homework assignments with other students in the class, as long as the following rules are followed:
    • You view another student’s work only for the purpose of offering/receiving assistance.
    • All work must be done by you.
    • Your discussion is subject to the empty hands policy, which means you leave the discussion without any record [electronic, mechanical, or otherwise] of the discussion.
  • Any material from any outside source such as books, projects, discussions with other students, and in particular, from the Web, should be properly referenced and should only be used if specifically allowed for the assignment.

8.3.3 Addendum

All code and problem solutions written for this class are the property of the course staff, who reserve all rights regarding such code and/or solutions. The course staff reserves the right, for example, to regularly pass your code and homework responses through online plagiarism checking services.

8.4 Additional Resources

he Colorado School of Mines is committed to ensuring the full participation of all students in its programs, including students with disabilities. If you are registered with Disability Support Services (DSS) and I have received your letter of accommodations, please contact me at your earliest convenience so we can discuss your needs in this course. For questions or other inquiries regarding disabilities, I encourage you to visit disabilities.mines.edu for more information.

Mines and the teaching staff of CSCI 598Bstrive to create a learning environment that is welcoming to students of all backgrounds. If you feel unwelcome for any reason let us know (i.e., tell Tom Williams) so that we can work to make things better. If you feel uncomfortable talking to members of the teaching staff, consider reaching out to your academic advisor, department chair, or dean.

8.5 Feedback

Your thoughts and concerns on this course are important. You are encouraged to give feedback to the instructor throughout the term. As always students will be asked to fill out a course evaluation at the end of the term.