E-learn Weblog of Willem van Valkenburg

We continue to develop new MOOCs as part of our Open & Online Learning programme. This fall we have 9 new MOOCs that will start, next to the reruns of 16 other MOOCs. In total we have 64 MOOCs announced and a couple more in development.

• Programming for teachers
  Last year we introduced a programming in Scratch for kids, in September the accompanying course for teachers will start. This course will teach teacher how to teach programming to kids. The course is in Dutch.
  
  
• Solar Energy: Photovoltaic Technologies
  This is the second course of the MicroMaster programme Solar Energy Engineering. This course will get you insight into the main PV technologies in the current market.
  
  
• Managing Building Adaptation: a Sustainable Approach
  The biggest sustainability challenge for cities worldwide is adapting existing obsolescent buildings and making them future-proof. In this course, you will learn about adapting buildings for sustainability.
  
  
• Business Model Metrics and Advanced Tools
  This course is the fifth course in the XSerie on Business Model Innovation. In this course you will learn advanced business model tools and metrics to help you achieve an agile business model.
  
  
• Co-Creating Sustainable Cities
  This is the second course we are co-developing with Wageningen University. This course is about how citizens can be co-creators of sustainable cities when they engage in city politics or in the design of the urban environment and its technologies and infrastructure.
  
  
• Railway Engineering: An Integral Approach
  Discover the science and complexity behind the exciting world of metro, tram and railway systems.
  
  
• Forensic Engineering: Learning from Failures
  Don’t let good failures go to waste! Identify the causes of failure and use this knowledge to enhance safety and improve performance.
  
  
• Globally Distributed Software Engineering
  Software engineering operates ever more frequently in globally distributed settings. This practice, also known as Globally Distributed Software Engineering (GDSE) has advantages and disadvantages that will be addressed in this course.
  
  
• Solar Energy: Photovoltaic (PV) Systems
  This is the third course of the MicroMaster Solar Energy Engineering. In this course you will explore the wide range of solar energy applications and learn to design a real PV installation with excellent performance and reliability.

Our MOOCs keep attracting new learners. This summer we reached the milestones of 1.5 million enrolments and 1 million learners. Overview of all the course we offer you can find on our online learning website.

Last month researchers from the University of Washington published an article about MOOC usage for professional workforce development outcomes in Colombia, the Philippines, & South Africa. There were a couple of news sites that used this article to say that students from developing countries have a high completion rate in MOOCs. To me that didn't seem like a logical conclusion, so I asked our very smart data analyst Sara Topolovec if we could see the same behaviour in our data set of MOOC enrolments.

There are two sets of data to look at:

• our surveys include questions that are similar to those of the UW research
• platform data

Survey data

If we look at the survey data (for 10 courses), the results are:

• 80% of both groups of students (developing and developed countries)  reported they had taken an online course before;
• 12% of students from developed countries that had taken an online course before, never completed any course, and 16% from developing, which means that
• 88% of students from developed countries that had taken an online course before state they completed at least one course, and 84% of students from developing countries.

Here we see that many students actually completed at least one course in both groups, but overall percentages are still slightly “in favour” of students from developing countries.

Platform data

If we look at the platform data (example of one course):

• The average grade of developing countries is 3,73% vs. 5,55% for developed, and the passing rate is 2,93% vs. 5,26%.
• There are more students from developing countries that hadn’t really started the MOOC (i.e. had a grade = 0), 91% vs. 87%, but even among those that did (grade > 0), the average grade is lower (36% vs 40%), as well as passing rates (32% vs. 47%).
• Only when you look only at people who received a certificate, the average grade is basically the same.

Because Sara is very good, she also had some comments about shortcomings of the original research:

• It tries to compare the result of their survey about completing any course ever to per-course completion rates. While it acknowledges that they don’t really have the actual data for comparison, it is still misleading to mention it alongside it, because in reality, it doesn’t tell us anything. They have no idea how many students in general completed at least one course already in developed countries.
• The comparison is problematic even further because of very different sources of information. While the completion rates are actual, true numbers, the survey is an estimate, that can be largely biased. We know that more students that receive a certificate in the end complete even the pre-survey (compared to the actual passing rates), which may also be true in this case, i.e. more engaged students completing their survey.
• Also, they compare their “completion” to per-course “completion rates”. But, our completion rates are actually certification rates, while their “completion” is completing the course with not necessarily receiving a certificate. Furthermore, students may understand very differently, what does it mean to complete a course, possibly connected to their intentions. We have no idea how many students in our courses would say they completed the course. So this is another point why these numbers can hardly be compared to the regular “completion rates”.
• On a per-course basis, the number of “registrants” is rather high (usually only around 50% of students does anything in our courseware), which is very far from the 2% of “registrants” they identified in their sample, which further shows that their completion numbers and course completion numbers can hardly be comparable.
• Their study is conducted only on people between 18-35 years old. As we know there are many students above 30 (30 is usually the median) in our courses, this is hardly representative of all MOOC users.

Conclusion

The researchers based their conclusion solely on self-reported survey data, but tried to compare their result to actual per-course completion rates, which creates a false sense that students from developing countries actually complete more courses than their peers from developed countries. While the high completion rates among students from developing countries may still be a surprise, it is important to keep in mind what we are actually looking at. Both platform data and survey data of ours revealed that somewhat more students from developed countries complete courses, or receive a certificate.  In most of our research we combine the survey data with the platform data to get more accurate results and less user bias.

Earlier this week Joshua Kim published a blog about 5 ways online teaching benefits residential learning. His 5 ways are:

1. A Chance to Rethink What Should Happen In the Physical Classroom
2. An Opportunity to Introduce Learning Theory and Pedagogical Research
3. Exposure to a Team Approach for Course Design
4. An Easier Transition to Blended Learning
5. Online Courses Help Make Teaching Strategies Visible to Educators and Students

These 5 ways are also true for my university. We have seen with a couple MOOCs that is started a complete change of the regular bachelor programme.

A good example is our mooc pre-university calculus. This course ran this summer with 27.000 students. The group that created the MOOC has also started a new programme (article in Dutch) to change the math education for all bachelor programmes. And they created a new website math-explained.tudelft.nl to make their video widely available for any student, but also for other teachers to include in their courses as reference material. And we got confirmation from other teachers that they use the content in their courses.

What we are further investigation is what the impact is on the pass rates for math in the first year. Traditionally the math courses are a hurdle for many students in passing the first year. In the survey the students really appreciated the course as is shown in the wordle above, where student were asked to give a one-word description.

Yesterday I attended the presentation of Dan Wagner about MOOCs4D and the new SDGs at the ICDE Conference. Now you think what the heck is he talking about. MOOCs4D is an initiative of University of Pennsylvania about MOOCs for Development, with a special focus on the bottom of the pyramid. SDG are the recently declared Sustainable Development Goals of the UN.

One of the things what triggered me where those 17 goals and how they match up with our TU Delft MOOCs. One of the objectives of our Open and Online Education programme is "Contribute to the Grand Challenges". I think we have to rephrase that to "Contribute to the SDGs". Back to our MOOCs, I could directly match 10 of our 20 MOOCs to 6 SDGs. In the table below the break-down of it.

We have some more MOOCs in development, but I decided not to add them yet. And there are also MOOCs that have a more general subject, such as our Leadership for Engineering and Creative Problem Solving and Decision Making.

Goal	MOOC
	Introduction to Drinking Water Treatment Learn about urban water services, focusing on conventional technologies for drinking water treatment. Introduction to the Treatment of Urban Sewage Learn about urban water services, focusing on basic sewage treatment technologies.
	Solar Energy Discover the power of solar energy and learn how to design a complete photovoltaic system. Wind Energy in development
	The Next Generation of Infrastructure Explore the challenges and complexity of both global and local infrastructure (IT/Telecom, Energy, Water and Transportation) and how to make the best decisions to improve it. Industrial Biotechnology Learn the basics of sustainable processing for biobased products to further understand their impact on global sustainability.
	Sustainable Urban Development: Discover Advanced Metropolitan Solutions Learn why cities are key in resolving global urbanization and sustainability challenges and how you can engineer tomorrow’s cities today.
	Responsible Innovation Responsible Innovation – exploring the interface of ethics, society and technological innovations Circular Economy: an introduction Learn how to contribute to a sustainable economic system by implementing novel business and design approaches
	Introduction to Water and Climate Explore how climate change, water availability, and engineering innovation are key challenges for our planet.

I also left out Goal 4: Ensure inclusive and quality education for all and promote lifelong learning. Because that includes all our MOOCs and our online courses for lifelong learners.

Do you want to be part of innovating education with online education? Help us to become a world leader in online education? Educate the world! We are looking for 2 eLearning Developers to join our (international) team!

TU Delft

Delft University of Technology is the leading University of Technology in the Netherlands with a global reputation in the fields of science, engineering & design. TU Delft’s mission is to make a significant contribution towards a sustainable society for the twenty-first century by conducting ground-breaking scientific and technological research which is acknowledged as world-class, by training scientists and engineers with a genuine commitment to society and by helping to translate knowledge into technological innovations and activity with both economic and social value.

Open & Online Education

TU Delft has been a forerunner in the field of Open & Online Education since 2007, and is one of the sustaining members of the Open Education Consortium and since 2013 charter member of the EdX Consortium. In March 2014, the board of the TU Delft approved a 3-year innovation program to accelerate the development of open & online education. The goal is to educate the world by creating an online learning portfolio with open & online courses for a global population of life long learners. Our courses are listed on https://online-learning.tudelft.nl.

Job description

As eLearning Developer you will be part of the (international) Extension School team, which supports and leads the online learning developments of the university. You will create advanced eLearning modules that will be part of the TU Delft Online Learning portfolio. The Extension School delivers MOOCs, Professional Education, blended and online education (bachelor and master level) to a worldwide audience. You will have considerable experience designing and developing online & blended education courses.

In this role you will partner with and support the academic staff to create outstanding open and online courses. Your project management skills contribute to your success in this role as you will be responsible for managing multiple projects. Together with a team of eLearning developers you will also work on constantly improving course designs and deliveries.

• Devise complex technical methods and processes to meet new and unique eLearning and web requirements and resolve technical problems.
• Ability to work independently. Ability to interact effectively with various faculty members in the TU Delft organization.
• Recognize, recommend and participate in the development of standards and procedures that support quality improvement in the Delft Extension School and products for our students.
• Provide project management for e-learning projects with tight deadlines. These projects bring together people from various departments and backgrounds to develop content, technical specifications and functional prototypes for the Extension School.
• Provide training to faculty on developing and teaching online courses.
• Meet all internal deadlines according to project plan regardless of project limitations.

Requirements

• Master of Education (M.Ed) in Instructional Design, Educational Technology or related experience plus 1 year recent curriculum design and/or course development experience in online education;
• Demonstrate a keen sense of organization and attention to detail;
• Demonstrated ability to handle unforeseen difficulties and obstacles with ease;
• An excellent team player that is able to work on complex projects with a group of other course developers;
• Ability to work on a tight, rapidly-changing schedule;
• Experience with using ICT in education, such as digital assessment, e-moderating, online teaching methods and platforms;
• A committed consultant with strong communication skills in English, mastery of Dutch is a plus;
• Experience in higher education;
• Experience as a teacher of open or online education is a plus.

For more information and application, please visit our official vacancies website. If you have questions you can contact me via contact form above.

TU Delft is a strong advocate for open education. So when we started with MOOCs on EdX.org, we published all the course materials with an open license. After the course is done, we publish the content also on our OpenCourseWare website.

Last year during the Open Education Week, I organised an Edx hackathon for our students. The winners of the hackathon were the guys of FeedbackFruits:

FeedbackFruits is a TU Delft startup that created an online study community that encourages students to participate in lectures and share study material to facilitate the core of education: opening up young minds to the wonders of science and helping students become specialists in the things they love.

FeedbackFruits received the award at the EdX conference in Delft in June last year. On stage I also requested EdX to include this plugin into the platform and they agreed. Unfortunately it took quiet some time to get the plugin included, but last Thursday it finally happened.

The plugin that was created allows course teams to set a Creative Commons license for the course and per item. For instruction, see the manual page.

Below are two screenshots to show what it looks like.

Further development

This plugin is the first step in implementing Creative Commons into the platform. Off course we have a wish list for improvements:

• remove the ND, because that is not really open.
• set the default to CC-BY.
• be able to attribute others for the content.

Other blogs

• Creative Commons
• OpenEdX

We started a blog to present the research of TU Delft Online Learning. On the blog we have published our working papers, conference papers and other articles. Please take a look at http://onlinelearningresearch.weblog.tudelft.nl.

Working Papers

We have published the first 6 working papers. The first five are course reports of our MOOCs in 2013-2014: Solar Energy, Water Treatment, Aeronautical Engineering, Credit Risk Management, and Next Generation Infrastructures.
The sixth report is a cross-course comparison of the first five DelftX MOOCs. The purpose of this analysis is to gain insight in the new opportunities and the new sources of data of this emerging online learning setting. The ultimate goal is to gain insights for enhancing DelftX educational provisions.

It has been a while that I have gave you an update about our MOOC activities. A lot of information I shared with the people that attended the Action Lab at OEGlobal, but that session wasn't recorded. So here is the update.

MOOCs

Since we began in 2013 we have developed 18 MOOCs. Some MOOC have run for multiple times. In total we already have more than 415,000 enrolments and have issued more than 12,000 certificates (1062 ID-verified).

New audiences

The first MOOCs we run were aiming at Bachelor students. This year we added some MOOCs that will attract other audiences. For example, the course Topology of Condensed Matter is much more directed toward Master and PhD students with a background in physics. We didn't expect a lot of learners, but still got more than 3,000. 50% of the learners has an advanced degree (Master or higher). Interesting is that the average age is lower than normal (27 instead of 29 in most courses).
Another course is the pre-university calculus course that focuses on high school students. 40% of the learners has a high school diploma as highest diploma. This course will start in July and the goal is to freshen up their math before they start at college or university in September. Already more than 10k learners signed up.

Self-Paced Courses

This summer we are offering two of our courses as self-paced courses starting on June 2nd. This means that there is almost now involvement of the teacher, but we do have student assistants moderating the forums. The courses we are offering are:

• Introduction to Aeronautical Engineering
• Creative Problem Solving and Decision Making

Translations

By default we offer English transcript. For the course Data Analysis to the MAX() we also translated the transcripts to Dutch, Hindi and Mandarin.

Because we also publish our MOOC content with an open license on ocw.tudelft.nl, the course Water Treatment is also translated to Japanese.

Sublicensing

�We are strong on openness, but also have to earn some money to cover our costs. Key questions are:

• Is it possible to sublicense a MOOC and uphold our Open Policy?
• Is our Open Policy threatened when we sublicense our MOOCs?�

Most sublicensing models are based on the licensing of content, but that doesn't work with our open license model. My colleague Martijn presented our solution at the OEGlobal Conference. I think we found a good solution that combines our open policy with sublicensing. The main slide is below:

Basically it means that our course materials are all openly-licensed, but you need a license with us if you want our Educational Services and Teacher Effort.

Based on this model, we got a sublicensing deal with Arabic MOOC platform EdRaak and there is interest from other organisations.

Online Learning Experiences

As part of our continues activities to enhance the quality of our courses we developed the online learning experience. The Online Learning Experience (OLE) is a student-centred, online learning model that holds eight interrelated principles, which define TU Delft�s online courses.

In the presentation below the model is explained:

At the EDEN Conference in Barcelona (Friday, parallel session H1) Nelson Jorge, Sofia Dopper and I will present the model in more detail, based on the paper we have written about it.

Future developments

There are more developments, but those I can't publish yet. So watch this space for more updates.

On May 26 2015, the Dutch Vereniging voor Onderwijs Research (VOR) organizes a symposium on research on MOOCs. During a number of presentations you will get an overview of current research into MOOCs and during the afternoon you will have the opportunity to listen to short case presentations by participants regarding research into MOOCs. But you can also provide your own cases to present at the symposium. There is an option on the registration form to indicate you interest in presenting a case.

One of the speakers is my colleague Pieter de Vries. He will talk about who's is the learner:

The Delft University of Technology (TUD) deployed her first generation of massive open online courses (MOOCs) in 2013-2014 delivered through the edX platform. These DelftX MOOCs were engineering courses designed at the level equivalent to that of a bachelor-program entry level. Almost 140 thousand students registered, around 3,7% received certificates of completion, and the rest participated to a degree reflective of their needs. To better understand the experiences of this massive number of learners, and ultimately enhance the MOOCs, TUD conducted the collection and analysis of data about learners and their contexts. This exploratory research focuses on the specific analyses pertinent to describing the demographics of an Engineering MOOC participant, as observed in the first generation of TUD MOOCs. The implications of the observed participant demographics are analysed and discussed.

The symposium will be on the campus of Wageningen University. For full programme and registration follow this link.

Yesterday it was my turn to do an action lab on MOOCs. The subtitle was "everything you want to know about MOOCs". For my it was also a good activity to get all our activities together. And it is impressive!

We have 16 MOOCs finished, 4 are running (you can still enrol), 5 new and 7 reruns are in production. So we can say that we have some experience with making MOOCs. We also have more than 400,000 enrolments.

I started the action lab with a short presentation about the TU Delft MOOC activities. The presentation is below. After the slide 21 I had some spare slide I could use to answer questions.

Journey

After the presentation I opened it for questions and used our MOOC Development journey to answer questions. I also had the poster of the learning experience flow of our award winning MOOC Delft Design Approach up on the wall. They used this to design their MOOC. They start with the template and use the cards to organise it week by week. All the documents are attached at the bottom of this post.

Sublicensing MOOCs

One of the questions I got was about the sublicensing of MOOCs. My colleague Martijn Ouwehand did a presentation about this subject on Wednesday. His slides are below.