{"id":268,"date":"2012-08-15T07:46:34","date_gmt":"2023-05-22T08:01:24","guid":{"rendered":"https:\/\/www.lasselaursen.com\/projects\/icon-set-selection-via-human-computation-2\/"},"modified":"2023-06-02T08:05:47","modified_gmt":"2023-06-02T08:05:47","slug":"social-dj","status":"publish","type":"page","link":"https:\/\/www.lasselaursen.com\/projects\/social-dj\/","title":{"rendered":"DJ Interface with Remote Audience Feedback"},"content":{"rendered":"\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>Streaming live performances is common place these days. While the technology to deliver the performance has massively improved over the past years, the methods with which to connect the performer with the audience have not undergone the same evolution. The most common place forms of audience\/performer interaction in a remote setting remains a standardized chat system, or possibly live poll.<\/p>\n\n\n\n<p>This project aims to explore novel ways of integrating the audience in a performance context, by displaying audience feedback within a performers software interface. We chose to focus on discjockeys (DJs), as audience feedback is key during their performances. DJs rely heavily on direct and indirect feedback from an audience to make informed decisions during their performances, such as which songs to play.<\/p>\n\n\n\n<p>With a remote DJs performance, meaning not in the same location as the audience, a lot of natural feedback disappears, such as the ability to see or hear the audience. This project explored providing DJs in a remote context with direct audience feedback, displayed in the software they used to perform. The user study revealed that while the performers were interested in the feedback they felt less inclined to take it into consideration, compared to a local performance.<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"A Multi-Touch DJ Interface with Remote Audience Feedback\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/2797ICjxY8w?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">Publication and resources<\/h4>\n\n\n\n<ul>\n<li><strong>A Multi-Touch DJ Interface with Remote Audience Feedback<\/strong><br>Lasse Farnung Laursen, Masataka Goto, Takeo Igarashi, in ACM MM \u201914: The 22nd ACM International Conference on Multimedia, pp. 1225-1228<br><a rel=\"noreferrer noopener\" href=\"https:\/\/dl.acm.org\/doi\/10.1145\/2647868.2655065?cid=81554950756\" target=\"_blank\">Published PDF<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/acmmm-poster-dj.pdf\" data-type=\"attachment\" data-id=\"124\">ACM MM &#8217;14 Poster<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.pechakucha.org\/presentations\/modern-touch-djing\">Pecha Kucha Presentation Modern Touch Djing<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.planmixplay.com\">PlanMixPlay<\/a> &#8211; Related future performance software<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">Quick Facts<\/h4>\n\n\n\n<ul>\n<li>Performance software built in C++ in one year. Progenitor project for PlanMixPlay.\n<ul>\n<li><a href=\"http:\/\/libcinder.org\">Cinder<\/a> used as base framework with <a href=\"http:\/\/www.un4seen.com\/bass.html\">BASS<\/a> as the audio library.<\/li>\n\n\n\n<li>Input and gesture detection system built from scratch supporting standard tap, drag, and pinch gestures.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Two formal user studies involving <a href=\"http:\/\/www.lasselaursen.com\/project\/social-dj\/#links\">7 DJs<\/a> conducted as part of project.<\/li>\n\n\n\n<li>Four 2-hour live shows also conducted as part of user study, with an average listener-count of 120 on <a href=\"http:\/\/www.slayradio.org\">Slayradio.org<\/a>.<\/li>\n\n\n\n<li>MT3200 touch-screen used as input device, produced by <a href=\"http:\/\/www.flatfrog.com\">FlatFrog<\/a>. The MT3200 was discontinued in 2014.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">DJ Interface<\/h4>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-2 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>A novel performance interface was developed to support feedback integration as part of this project. The novel interface consists of an audio time-line visualization, with general playback and visualization controls at the bottom and a set of drop-down menus for loading audio files. The entire time-line supports pinch gesture zooming for easy overview or detailed inspection. All loaded audio elements support intuitive drag-and-drop style interaction. Two modes of visualized playback are supported; static, where the time-line is still and the playback line moves, or moving time-line with a static playback line. The former is ideal for planning ahead and setting up audio elements in advance. The latter mode of visualization is a more dynamic \u2018on-the-fly\u2019 style interaction mode.<\/p>\n\n\n\n<p>The MT3200 touch-screen is used as an input device. The touch-screen itself supports up to 41 simultaneous touches, as does the DJ interface. The large size of the screen easily allows two users to simultaneously manipulate the time-line and its contained audio elements.<\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<figure class=\"wp-block-image size-large\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface.png\" data-rel=\"lightbox-gallery-22uk4dA5\" data-magnific_type=\"image\" data-rl_title=\"Performance Interface - Tested iteration.\" data-rl_caption=\"A screen-shot of the interface used in the 4 two-hour live shows (shown here with mock-feedback). This version supports simplified positive feedback, along with user comments and a listener histogram. All of the feedback can be toggled on\/off at any time during a performance. The system distinguishes between two types of listeners (gold \/ bronze) who represent those known and not-known to the DJ, respectively.\" title=\"Performance Interface - Tested iteration.\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"576\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface-1024x576.png\" alt=\"\" class=\"wp-image-445\" srcset=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface-1024x576.png 1024w, https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface-300x169.png 300w, https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface-768x432.png 768w, https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface-1536x864.png 1536w, https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><figcaption class=\"wp-element-caption\">Performance Interface &#8211; Tested iteration.<\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">Postmortem<\/h4>\n\n\n\n<p>Longer projects, 3-months or more, tend to invite more introspection than shorter ones. Did I make the right design choices? What could I have done better? Regrets are typically a waste of energy, but self-assessment is necessary to improve. We must analyze our past decision-making, lest we fall into the all too familiar trap <a href=\"https:\/\/www.youtube.com\/watch?v=3iFxUCSTfRU\" data-rel=\"lightbox-video-0\" data-magnific_type=\"video\">of thinking we made all the right moves<\/a>, because we were the ones making them&#8230;<\/p>\n\n\n\n<p>From the onset of the project, a small set of design requirements were determined:<\/p>\n\n\n\n<ul>\n<li><strong>Fast real-time performance<\/strong> &#8211; The system <strong>must<\/strong> be responsive. Compared to live performance instruments, touch-screens (especially in 2014) were incredibly laggy. Improving input hardware response times was well beyond the scope of the project, so the next best thing was to ensure the software didn&#8217;t aggravate the issue by introducing further lag.<\/li>\n\n\n\n<li><strong>Robust<\/strong> &#8211; The system must be stable enough to execute continuously for several hours with user interaction, without crashing.<\/li>\n<\/ul>\n\n\n\n<p>Given these requirements, I opted to build the software in C++. In hindsight, given the prototypical nature of the project, I wonder if Python or another higher level language would have served better. Python, or similar alternative, would have allowed for faster design iteration meaning more time to test multiple feedback mechanisms. However, Python could have presented functional barriers which would have required low-level programming regardless. E.g., in a recent personal project I got burned a bit by Python and its collection of audio libraries as I was attempting to manually synchronize two audio streams. The audio libraries (at the time) didn&#8217;t allow direct writing of data to the audio buffer and thus various workarounds had to be attempted, none of which ended up working satisfactorily.<\/p>\n\n\n\n<p>Another design decision I&#8217;ve pondered it the decision to build my own GUI library. In 2012 when the project commenced, only a limited number of GUI libraries for C++ featured touch input for Windows applications. Building my own GUI library taught me a lot, allowed me to seamlessly integrate keyboard-, mouse-, and touch-input, but also turned out to be very time consuming. The input portion alone took at least 3-4 months to reach its final design iteration. In hindsight, I still think this was probably necessary, as adding touch interaction to a GUI library that only supports mouse and keyboard is no small feat. A mouse always represents a singular point of interaction. Touch interaction is akin to one mouse per each supported touch, which in the case of the MT3200 was 41. A well designed GUI library supporting only mouse and keyboard would require substantial refactoring to properly support multiple simultaneous &#8216;mice&#8217;.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">Observations and results<\/h4>\n\n\n\n<p>Here are the most interesting findings derived from this project in regards to integrated remote audience feedback:<\/p>\n\n\n\n<ul>\n<li><strong>Preference for comments<\/strong> &#8211; The DJs generally preferred comments to likes, as they felt it showed more of an effort on behalf of the audience member.<\/li>\n\n\n\n<li><strong>Filter use was non-existant<\/strong> &#8211; The DJs never filtered any of the feedback at any point during the two-hour shows. However, this is likely due to the interface being very simple overall, thus making the feedback not intrusive.<\/li>\n\n\n\n<li><strong>Application in a local setting<\/strong> &#8211; Some DJs expressed an interest in having this type of feedback available during a local performance.<\/li>\n\n\n\n<li><strong>No strong differentiation between feedback types<\/strong> &#8211; Overall, the DJs did not seem to value the feedback differently depending on the type (gold\/bronze).<\/li>\n<\/ul>\n\n\n\n<p>The participating DJs also made a number of interesting observations:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>The feedback was helpful. Particularly if I was doing an online set, that would be really useful. [The feedback] is more direct. If we use a dance floor analogy; People will leave the dance floor [due to fatigue or to get a drink,] not cause they don\u2019t like the music [\u2026]. [\u2026] So in that way, dancing is kind of an imprecise metric.<\/p>\n<cite>&#8211; DJ Modality<\/cite><\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>I wouldn\u2019t do a live [remote] mix, and then get some negative feedback, and [\u2026] just switch genres. I might do that at a [local] club.<\/p>\n<cite>&#8211; Philip Huey<\/cite><\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>It gives you inspiration and energy. [\u2026] It works. I was surprised.<\/p>\n<cite>&#8211; Mark Jackson<\/cite><\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>It\u2019s interesting to see how people react and what they do. Will it heavily influence what I do? I don\u2019t think so.<\/p>\n<cite>&#8211; Matthew Gammon<\/cite><\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>It would help me understand my audience better, know what they want to hear, which could lead to better parties or sets. That\u2019s important. [\u2026] I can see a lot of uses for that.<\/p>\n<cite>&#8211; DJ Vivid<\/cite><\/blockquote>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">Gallery<\/h4>\n\n\n\n<div class=\"wp-block-responsive-lightbox-gallery\"><div class=\"rl-gallery-container rl-loading\" id=\"rl-gallery-container-1\" data-gallery_id=\"438\"> <div class=\"rl-gallery rl-basicgrid-gallery \" id=\"rl-gallery-1\" data-gallery_no=\"1\"> <div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/early-prototype.png\" title=\"Performance Interface - Early Iteration\" data-rl_title=\"Performance Interface - Early Iteration\" class=\"rl-gallery-link\" data-rl_caption=\"A very early prototype screen-shot featuring a basic time-line and sparse control functionality.\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/early-prototype-300x169.png\" width=\"300\" height=\"169\" alt=\"\"\/><\/a><\/div><div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/early-feedback.png\" title=\"Performance Interface - Early Iteration 2\" data-rl_title=\"Performance Interface - Early Iteration 2\" class=\"rl-gallery-link\" data-rl_caption=\"Another early prototype screen-shot showcasing some integrated audience feedback. Initially, audience members could only give general positive\/negative feedback (like\/dislike) and positive\/negative feedback with regards to the volume (too loud \/ too quiet).\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/early-feedback-300x169.png\" width=\"300\" height=\"169\" alt=\"\"\/><\/a><\/div><div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface.png\" title=\"Performance Interface - Tested iteration.\" data-rl_title=\"Performance Interface - Tested iteration.\" class=\"rl-gallery-link\" data-rl_caption=\"A screen-shot of the interface used in the 4 two-hour live shows (shown here with mock-feedback). This version supports simplified positive feedback, along with user comments and a listener histogram. All of the feedback can be toggled on\/off at any time during a performance. The system distinguishes between two types of listeners (gold \/ bronze) who represent those known and not-known to the DJ, respectively.\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/Interface-300x169.png\" width=\"300\" height=\"169\" alt=\"\"\/><\/a><\/div><div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/early-audience-interface.png\" title=\"Audience interaction interface - early iteration.\" data-rl_title=\"Audience interaction interface - early iteration.\" class=\"rl-gallery-link\" data-rl_caption=\"An early screen-shot of the audience feedback interface. It consisted of only four buttons enabling the simple discreet feedback to be sent directly to the DJ interface.\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/early-audience-interface-296x300.png\" width=\"296\" height=\"300\" alt=\"\"\/><\/a><\/div><div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/audience-interface.png\" title=\"Audience interaction interface - Tested iteration.\" data-rl_title=\"Audience interaction interface - Tested iteration.\" class=\"rl-gallery-link\" data-rl_caption=\"A screen-shot of the audience (web-based) interface used in the 4 two-hour live shows (shown here with mock feedback). The users could actively participate by sending both a discrete &#039;like&#039; or a comment.\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/audience-interface-300x258.png\" width=\"300\" height=\"258\" alt=\"\"\/><\/a><\/div><div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/broadcast-setup.jpg\" title=\"Interface setup in Lab.\" data-rl_title=\"Interface setup in Lab.\" class=\"rl-gallery-link\" data-rl_caption=\"The broadcasting setup as used in each of the 4 two-hour live shows at the University of Tokyo, Igarashi Design Lab.\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/broadcast-setup-300x225.jpg\" width=\"300\" height=\"225\" alt=\"\"\/><\/a><\/div><div class=\"rl-gallery-item\"><a href=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/collage.jpg\" title=\"Commemorative collage.\" data-rl_title=\"Commemorative collage.\" class=\"rl-gallery-link\" data-rl_caption=\"A commemorative collage of many of the participants who helped test the DJ interface during its early development.\" data-rel=\"lightbox-gallery-1\" data-magnific_type=\"gallery\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.lasselaursen.com\/wp-content\/uploads\/2023\/05\/collage-300x169.jpg\" width=\"300\" height=\"169\" alt=\"\"\/><\/a><\/div> <\/div> <\/div><\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\">Acknowledgments<\/h4>\n\n\n\n<p>This project wouldn&#8217;t have come to fruition if it wasn&#8217;t for the help of both my supervisors <a href=\"http:\/\/staff.aist.go.jp\/m.goto\/\">Masataka Goto<\/a> and <a href=\"http:\/\/www-ui.is.s.u-tokyo.ac.jp\/~takeo\/\">Takeo Igarashi<\/a>, Slaygon and SlayRadio.org, as well as all of the participating DJs.<\/p>\n\n\n\n<ul>\n<li><a href=\"http:\/\/www.slayradio.org\">Slayradio.org<\/a><\/li>\n\n\n\n<li>Participating DJs\n<ul>\n<li><a href=\"http:\/\/soundcloud.com\/nytelyfe\">Phil Huey<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www-ui.is.s.u-tokyo.ac.jp\/~nakajima\/index_en.html\">Makoto Nakajima<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/soundcloud.com\/djmodality\">Dj Modality<\/a><\/li>\n\n\n\n<li>Mark Jackson<\/li>\n\n\n\n<li><a href=\"http:\/\/soundcloud.com\/telquel\">Matthew Gammon<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.thevividexperience.com\/\">Dj Vivid<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/soundcloud.com\/djhasai\">Jason Hasai<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Streaming live performances is common place these days. While the technology to deliver the performance has massively improved over the past years, the methods with which to connect the performer with the audience have not undergone the same evolution. The most common place forms of audience\/performer interaction in a remote setting remains a standardized chat [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":556,"parent":142,"menu_order":4,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_FSMCFIC_featured_image_caption":"","_FSMCFIC_featured_image_nocaption":"","_FSMCFIC_featured_image_hide":"","footnotes":""},"_links":{"self":[{"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/pages\/268"}],"collection":[{"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/comments?post=268"}],"version-history":[{"count":10,"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/pages\/268\/revisions"}],"predecessor-version":[{"id":447,"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/pages\/268\/revisions\/447"}],"up":[{"embeddable":true,"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/pages\/142"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/media\/556"}],"wp:attachment":[{"href":"https:\/\/www.lasselaursen.com\/wp-json\/wp\/v2\/media?parent=268"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}