WEBVTT 00:00:00.000 --> 00:00:05.700 align:middle line:90% 00:00:05.700 --> 00:00:06.200 align:middle line:90% OK. 00:00:06.200 --> 00:00:09.780 align:middle line:84% In the last section, we looked at the details of how 00:00:09.780 --> 00:00:11.580 align:middle line:90% we can do motion capture. 00:00:11.580 --> 00:00:14.520 align:middle line:84% How the cameras work, the calibration, and also 00:00:14.520 --> 00:00:15.690 align:middle line:90% some of the post-processing. 00:00:15.690 --> 00:00:17.190 align:middle line:84% In this section, we're going to look 00:00:17.190 --> 00:00:20.730 align:middle line:84% at some examples of how this type of technology 00:00:20.730 --> 00:00:23.070 align:middle line:90% is used in real research. 00:00:23.070 --> 00:00:25.410 align:middle line:84% And we're going to then talk to some of the researchers 00:00:25.410 --> 00:00:26.800 align:middle line:90% about their experiments. 00:00:26.800 --> 00:00:29.370 align:middle line:84% And we'll start out by looking at some examples 00:00:29.370 --> 00:00:31.142 align:middle line:90% of motion capture of musicians. 00:00:31.142 --> 00:00:32.850 align:middle line:84% You can do this by looking at individuals 00:00:32.850 --> 00:00:37.290 align:middle line:84% or by looking at multiple performers. 00:00:37.290 --> 00:00:40.660 align:middle line:84% Laura, you have experience with doing this. 00:00:40.660 --> 00:00:44.340 align:middle line:84% And in the case of what we did in the library 00:00:44.340 --> 00:00:46.835 align:middle line:90% with a string quartet-- 00:00:46.835 --> 00:00:48.210 align:middle line:84% this was actually the second time 00:00:48.210 --> 00:00:51.240 align:middle line:84% we recorded this string quartet, because we already 00:00:51.240 --> 00:00:54.180 align:middle line:84% recorded them once before, here in the lab. 00:00:54.180 --> 00:00:56.400 align:middle line:90% Yes. 00:00:56.400 --> 00:00:58.650 align:middle line:84% What, in your experience, is the difference 00:00:58.650 --> 00:01:01.140 align:middle line:84% between doing such a recording in the lab 00:01:01.140 --> 00:01:03.630 align:middle line:84% and doing it in a public setting? 00:01:03.630 --> 00:01:06.750 align:middle line:84% Well, in a public setting, it requires a lot 00:01:06.750 --> 00:01:09.540 align:middle line:90% of on-the-spot preparation. 00:01:09.540 --> 00:01:12.570 align:middle line:84% In the lab, you have your cameras in place 00:01:12.570 --> 00:01:16.560 align:middle line:84% already, so you can kind of prepare in advance where 00:01:16.560 --> 00:01:17.873 align:middle line:90% the musicians are going to sit. 00:01:17.873 --> 00:01:19.290 align:middle line:84% And you know the cameras are going 00:01:19.290 --> 00:01:21.090 align:middle line:84% to be in a useful position, and you've 00:01:21.090 --> 00:01:22.860 align:middle line:90% got everything connected. 00:01:22.860 --> 00:01:25.290 align:middle line:84% But when you try to capture something 00:01:25.290 --> 00:01:30.002 align:middle line:84% in real life, like we did in the library, basically 00:01:30.002 --> 00:01:31.710 align:middle line:84% an hour or two before the musicians come, 00:01:31.710 --> 00:01:34.800 align:middle line:84% you have to put all the cameras in a useful place 00:01:34.800 --> 00:01:37.140 align:middle line:84% and try to point them so that they're 00:01:37.140 --> 00:01:38.640 align:middle line:84% facing the direction that you expect 00:01:38.640 --> 00:01:41.760 align:middle line:90% the musicians are going to be. 00:01:41.760 --> 00:01:44.400 align:middle line:84% This is not always very easy or very successful. 00:01:44.400 --> 00:01:47.580 align:middle line:84% And you might have to deal with some difficulties 00:01:47.580 --> 00:01:51.480 align:middle line:84% with the lighting, or reflections coming from-- 00:01:51.480 --> 00:01:55.410 align:middle line:84% if the floor is not ideal and kind of shiny, 00:01:55.410 --> 00:01:59.110 align:middle line:84% then this could also be a problem. 00:01:59.110 --> 00:02:02.670 align:middle line:84% So it requires a lot of on-the-spot troubleshooting. 00:02:02.670 --> 00:02:04.650 align:middle line:84% But the quality of the data is more or less 00:02:04.650 --> 00:02:06.840 align:middle line:90% the same, or what do you think? 00:02:06.840 --> 00:02:09.060 align:middle line:84% Yeah well actually, in both of these cases, 00:02:09.060 --> 00:02:10.330 align:middle line:90% it was kind of a trial run. 00:02:10.330 --> 00:02:13.770 align:middle line:84% So in neither case we actually had the ability 00:02:13.770 --> 00:02:16.140 align:middle line:84% to set up very much in advance and have 00:02:16.140 --> 00:02:19.680 align:middle line:84% the musicians in advance, so we could make sure that everything 00:02:19.680 --> 00:02:21.840 align:middle line:90% was going to be captured well. 00:02:21.840 --> 00:02:26.070 align:middle line:84% So the quality of the data ended up being similar in both cases. 00:02:26.070 --> 00:02:32.010 align:middle line:84% We had some issues with the backs of the second violinist 00:02:32.010 --> 00:02:34.080 align:middle line:84% and the violist not being captured well, 00:02:34.080 --> 00:02:37.110 align:middle line:84% because they ended up being placed a little bit too closely 00:02:37.110 --> 00:02:40.260 align:middle line:90% to the edge of the MoCap space. 00:02:40.260 --> 00:02:45.810 align:middle line:84% And so, for instance, in the library concert, 00:02:45.810 --> 00:02:47.490 align:middle line:84% now that we've seen this data and seen 00:02:47.490 --> 00:02:50.040 align:middle line:84% that this is the problem, next time we would know 00:02:50.040 --> 00:02:52.740 align:middle line:84% that we need to put cameras lower down, 00:02:52.740 --> 00:02:54.000 align:middle line:90% right behind the people. 00:02:54.000 --> 00:02:57.600 align:middle line:84% Maybe move the musicians forward a little bit. 00:02:57.600 --> 00:02:59.728 align:middle line:84% And I guess in both of these settings, 00:02:59.728 --> 00:03:01.770 align:middle line:84% we actually had also an audience involved, right? 00:03:01.770 --> 00:03:03.360 align:middle line:84% So even in the lab setting we decided 00:03:03.360 --> 00:03:06.000 align:middle line:84% to run it with an audience, and why was that? 00:03:06.000 --> 00:03:09.000 align:middle line:84% Well in this case, we were interested in how 00:03:09.000 --> 00:03:12.960 align:middle line:84% the musicians' behaviour differs between whether 00:03:12.960 --> 00:03:15.510 align:middle line:90% there's an audience or not. 00:03:15.510 --> 00:03:17.730 align:middle line:84% So when there's an audience, there's more pressure, 00:03:17.730 --> 00:03:18.630 align:middle line:90% it's more stressful. 00:03:18.630 --> 00:03:20.010 align:middle line:90% It's also more exciting. 00:03:20.010 --> 00:03:23.280 align:middle line:84% And so we were testing some hypotheses 00:03:23.280 --> 00:03:26.700 align:middle line:84% about how the musicians might respond to these conditions. 00:03:26.700 --> 00:03:30.750 align:middle line:84% And in particular, I was kind of expecting 00:03:30.750 --> 00:03:33.510 align:middle line:84% that they might be more interactive, more overtly 00:03:33.510 --> 00:03:34.798 align:middle line:90% interactive with each other. 00:03:34.798 --> 00:03:36.840 align:middle line:84% So in that case, we would see certain differences 00:03:36.840 --> 00:03:39.870 align:middle line:84% in the way they move, the amount that they move, 00:03:39.870 --> 00:03:42.497 align:middle line:84% and how their movements relate to each other. 00:03:42.497 --> 00:03:44.080 align:middle line:84% And you have some graphs here as well. 00:03:44.080 --> 00:03:45.960 align:middle line:10% So what do we see here? 00:03:45.960 --> 00:03:50.100 align:middle line:84% This is the trajectory of the marker that 00:03:50.100 --> 00:03:52.710 align:middle line:84% was placed on the back of the cello 00:03:52.710 --> 00:03:57.840 align:middle line:84% in the vertical direction, over the course of a performance 00:03:57.840 --> 00:04:01.420 align:middle line:90% of about two minutes. 00:04:01.420 --> 00:04:03.720 align:middle line:84% So this is actually measured in terms of the distance 00:04:03.720 --> 00:04:05.290 align:middle line:90% from the starting position. 00:04:05.290 --> 00:04:08.730 align:middle line:84% So you can see how much she changed her position 00:04:08.730 --> 00:04:10.950 align:middle line:84% over the course of the performance. 00:04:10.950 --> 00:04:12.900 align:middle line:84% Is that a type of-- do you often like 00:04:12.900 --> 00:04:14.400 align:middle line:84% to use that type of measure, instead 00:04:14.400 --> 00:04:16.487 align:middle line:90% of looking at the position? 00:04:16.487 --> 00:04:18.570 align:middle line:84% Actually I don't usually use this kind of measure. 00:04:18.570 --> 00:04:20.940 align:middle line:84% In this particular case, I used this measure 00:04:20.940 --> 00:04:22.740 align:middle line:84% because I was making a comparison 00:04:22.740 --> 00:04:25.620 align:middle line:84% of two different motion capture systems that 00:04:25.620 --> 00:04:29.040 align:middle line:84% were used to capture the same performance session. 00:04:29.040 --> 00:04:32.010 align:middle line:84% And because the cameras are in different positions 00:04:32.010 --> 00:04:34.230 align:middle line:84% than the actually absolute position 00:04:34.230 --> 00:04:37.100 align:middle line:84% values that we got for the markers were 00:04:37.100 --> 00:04:38.850 align:middle line:84% in different coordinate systems, and these 00:04:38.850 --> 00:04:40.710 align:middle line:90% had to be compared somehow. 00:04:40.710 --> 00:04:44.250 align:middle line:84% So taking the position as a distance 00:04:44.250 --> 00:04:48.540 align:middle line:84% from the starting position from the origin of the marker 00:04:48.540 --> 00:04:50.910 align:middle line:84% was a useful way to make that comparison. 00:04:50.910 --> 00:04:52.740 align:middle line:84% And then, since in this case, you actually 00:04:52.740 --> 00:04:55.230 align:middle line:84% did then study and use two different motion capture 00:04:55.230 --> 00:04:59.610 align:middle line:84% systems, what is your experience with the systems 00:04:59.610 --> 00:05:01.400 align:middle line:90% and how similar are they? 00:05:01.400 --> 00:05:03.610 align:middle line:90% They actually are very similar. 00:05:03.610 --> 00:05:05.670 align:middle line:84% It was quite interesting, because we had-- it 00:05:05.670 --> 00:05:06.920 align:middle line:90% was an interesting comparison. 00:05:06.920 --> 00:05:11.400 align:middle line:84% Because we had one very, very expensive top-of-the-line 00:05:11.400 --> 00:05:15.360 align:middle line:84% system, which was about 10 years old at the time that we made 00:05:15.360 --> 00:05:16.320 align:middle line:90% the capture. 00:05:16.320 --> 00:05:20.310 align:middle line:84% And then an almost brand new system, which was-- 00:05:20.310 --> 00:05:24.690 align:middle line:84% so the cameras are a lot smaller and more affordable, 00:05:24.690 --> 00:05:25.990 align:middle line:90% but was brand new. 00:05:25.990 --> 00:05:28.530 align:middle line:84% So it was actually very interesting to see 00:05:28.530 --> 00:05:32.620 align:middle line:84% that the systems performed almost the same. 00:05:32.620 --> 00:05:35.493 align:middle line:84% Really there was not too much difference. 00:05:35.493 --> 00:05:37.410 align:middle line:84% Normally, you would expect the high end system 00:05:37.410 --> 00:05:38.290 align:middle line:90% to perform better. 00:05:38.290 --> 00:05:42.390 align:middle line:84% But this is a 10-year-old system. 00:05:42.390 --> 00:05:44.700 align:middle line:84% And then, in terms of the noise, then 00:05:44.700 --> 00:05:46.833 align:middle line:84% that you can see in the systems, that's 00:05:46.833 --> 00:05:48.750 align:middle line:84% something that we have been interested in here 00:05:48.750 --> 00:05:50.950 align:middle line:84% because we also have studies where 00:05:50.950 --> 00:05:55.200 align:middle line:90% we look at smaller movements. 00:05:55.200 --> 00:05:57.090 align:middle line:90% What do we see here? 00:05:57.090 --> 00:06:02.580 align:middle line:10% This, in red and blue, this is the noise 00:06:02.580 --> 00:06:06.750 align:middle line:10% that we found in a small clip of a recording 00:06:06.750 --> 00:06:09.040 align:middle line:10% from a marker that was standing still. 00:06:09.040 --> 00:06:10.950 align:middle line:84% So this marker was actually placed 00:06:10.950 --> 00:06:13.380 align:middle line:84% on one of the music stands of one of the performers, 00:06:13.380 --> 00:06:16.360 align:middle line:84% and so it should not be moving at all. 00:06:16.360 --> 00:06:20.310 align:middle line:84% But even markers that are sitting perfectly still 00:06:20.310 --> 00:06:22.210 align:middle line:84% register some movement in the system. 00:06:22.210 --> 00:06:24.990 align:middle line:84% And this is what we refer to as noise. 00:06:24.990 --> 00:06:28.060 align:middle line:90% And this is unavoidable. 00:06:28.060 --> 00:06:30.600 align:middle line:84% So this graph shows the noise for the same marker 00:06:30.600 --> 00:06:32.290 align:middle line:90% from the two systems. 00:06:32.290 --> 00:06:35.220 align:middle line:10% And if you look at the y-axis, this 00:06:35.220 --> 00:06:38.050 align:middle line:10% is in millimetres, so this is fractions of a millimetre. 00:06:38.050 --> 00:06:40.175 align:middle line:10% So it's really actually a very, very small, number. 00:06:40.175 --> 00:06:43.390 align:middle line:10% Very, very tiny amount of movement. 00:06:43.390 --> 00:06:47.610 align:middle line:84% So even this is kind of big jump is not actually a big jump. 00:06:47.610 --> 00:06:51.828 align:middle line:90% This is 0.2 of a millimetre. 00:06:51.828 --> 00:06:53.370 align:middle line:84% This is interesting, because it shows 00:06:53.370 --> 00:06:55.140 align:middle line:84% the different kinds of noise that you 00:06:55.140 --> 00:06:56.770 align:middle line:90% can see in these systems. 00:06:56.770 --> 00:07:00.678 align:middle line:84% So you get a lot of this very, very small up and down jitter. 00:07:00.678 --> 00:07:02.220 align:middle line:84% And then sometimes you get these kind 00:07:02.220 --> 00:07:05.910 align:middle line:84% of spikes that are just jumps that jump back. 00:07:05.910 --> 00:07:08.130 align:middle line:84% And then you also get this kind of thing, 00:07:08.130 --> 00:07:12.270 align:middle line:84% where the whole marker position shifts a very small amounts. 00:07:12.270 --> 00:07:16.090 align:middle line:84% It says in the same place and then goes back to where it was. 00:07:16.090 --> 00:07:18.210 align:middle line:84% So I guess this is something that we often 00:07:18.210 --> 00:07:21.600 align:middle line:84% do in our studies, is that we like to have a still marker 00:07:21.600 --> 00:07:24.760 align:middle line:84% to be able to also measure the noise level of the system. 00:07:24.760 --> 00:07:26.198 align:middle line:84% It's good as a reference, I think, 00:07:26.198 --> 00:07:27.240 align:middle line:90% for these type of things. 00:07:27.240 --> 00:07:28.770 align:middle line:90% Yes. 00:07:28.770 --> 00:07:30.450 align:middle line:84% And then, if you were kind of trying 00:07:30.450 --> 00:07:32.970 align:middle line:84% to summarise all your experience with working with capturing 00:07:32.970 --> 00:07:35.380 align:middle line:84% musicians or pianists and string players, 00:07:35.380 --> 00:07:37.960 align:middle line:84% et cetera, what are kind of the most important things 00:07:37.960 --> 00:07:40.980 align:middle line:84% do you think to think about when you do that type of motion 00:07:40.980 --> 00:07:42.410 align:middle line:90% capture study? 00:07:42.410 --> 00:07:44.220 align:middle line:84% I guess you have to think very carefully 00:07:44.220 --> 00:07:47.010 align:middle line:84% about the type of motion that you want to capture. 00:07:47.010 --> 00:07:49.485 align:middle line:84% The parts of the body that you want to capture. 00:07:49.485 --> 00:07:52.680 align:middle line:84% And come to an agreement with yourself 00:07:52.680 --> 00:07:55.613 align:middle line:84% about which parts of the body you want to measure, 00:07:55.613 --> 00:07:57.030 align:middle line:84% and which parts are actually going 00:07:57.030 --> 00:08:00.430 align:middle line:90% to be measured successfully. 00:08:00.430 --> 00:08:05.820 align:middle line:84% So for instance, with the string musicians, 00:08:05.820 --> 00:08:08.790 align:middle line:84% you see that some of the body is covered by the instrument. 00:08:08.790 --> 00:08:11.310 align:middle line:84% So for instance, with pianists, maybe it's 00:08:11.310 --> 00:08:13.050 align:middle line:84% interesting to look at the shoulders. 00:08:13.050 --> 00:08:14.850 align:middle line:84% With a string musician, like a violinist, 00:08:14.850 --> 00:08:19.470 align:middle line:84% who's got their violin here, you can't really get this motion. 00:08:19.470 --> 00:08:21.330 align:middle line:84% And you can't really get the arm motion. 00:08:21.330 --> 00:08:24.585 align:middle line:84% So if it's important for you to have this, 00:08:24.585 --> 00:08:27.210 align:middle line:84% you need to choose whether it's important for you to have this, 00:08:27.210 --> 00:08:32.200 align:middle line:84% and then maybe you need to place the markers in a specific way. 00:08:32.200 --> 00:08:34.867 align:middle line:84% So then, what you're saying is that the marker placement 00:08:34.867 --> 00:08:36.659 align:middle line:84% is very important for these type of things. 00:08:36.659 --> 00:08:38.700 align:middle line:84% And particularly to also to consider the instrument. 00:08:38.700 --> 00:08:39.200 align:middle line:90% Yes. 00:08:39.200 --> 00:08:39.870 align:middle line:90% Yes? 00:08:39.870 --> 00:08:42.030 align:middle line:84% Yes, and it's also interesting to measure 00:08:42.030 --> 00:08:47.280 align:middle line:84% the movement of the instrument in many cases. 00:08:47.280 --> 00:08:48.840 align:middle line:84% I guess this is a bit philosophical, 00:08:48.840 --> 00:08:51.270 align:middle line:84% but to some extent, the instrument 00:08:51.270 --> 00:08:53.760 align:middle line:90% becomes part of the person. 00:08:53.760 --> 00:08:57.090 align:middle line:84% It's being controlled by the person, and for some musicians, 00:08:57.090 --> 00:08:59.940 align:middle line:84% this may be a kind of part of themselves 00:08:59.940 --> 00:09:02.400 align:middle line:90% as they're producing the music. 00:09:02.400 --> 00:09:04.530 align:middle line:84% So in that way, it's very interesting 00:09:04.530 --> 00:09:06.450 align:middle line:84% to look at how they are manipulating 00:09:06.450 --> 00:09:11.800 align:middle line:84% the instrument and the bow, if there's a bow involved. 00:09:11.800 --> 00:09:12.310 align:middle line:90% OK. 00:09:12.310 --> 00:09:15.850 align:middle line:84% So that's some good tips when it comes to capturing musicians. 00:09:15.850 --> 00:09:18.460 align:middle line:84% And we're going to then jump over 00:09:18.460 --> 00:09:22.950 align:middle line:84% to looking at also how we can capture perceivers. 00:09:22.950 --> 00:09:30.000 align:middle line:90%