Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Learning Analytics Dashboards

Learning Analytics Dashboards Chapter 12: Learning Analytics Dashboards HUEHUDOWDQG(ULN'XY-RULV.OHUN[.DWULHQ9 Department of Computer Science, KU Leuven, Belgium DOI: 10.18608/hla17.012 ABSTRACT 7KFKLVWSSUDHHUVWHQOHVDUQLDQJQDO\WLFGVDVKERDUGWVKYWDL VXDOLOH]HDUQLWQJUDFHWJVRHLY HODU(WURDS[FHHKQVDLVLWHOSQPQRVJLQXVLJVWUHKV GDWVKEGHRKDWVDHGUVDZKWDIRZRVKQUH LVDRJOWHKWKZDHGGQWHLQDHUWH\KPRKZURIXVH QWGRGLLDYQ,XLVDOL]HEGHFDQWGDDZRKDGQ guidelines on how to get started with the development of learning analytics dashboards are presented for practitioners and researchers. Keywords:,UQRIPWDQYLRLVXDOL]WDQLROHDUQLQJDQDO\WLFVGDVKERDUGV In recent years, many learning analytics dashboards BACKGROUND have been deployed to support insight into learning data. The objectives of these dashboards include To Augment the Human Intellect providing feedback on learning activities, supporting There is a strong contrast between intelligent systems QHPHDJJQHLJHQDVUFLQPDNLJQQRLVLFHGGDQQRFHWLĂHU W that try to make decisions on behalf of people, such and motivation, and reducing dropout. These learning as intelligent tutoring systems (Brusilovsky, 2000) DQDO\LGUQRWDYIVKLFSESODP\VRWDLDVXQLURGDVOL]WDQLR and educational data mining systems (Santos et al., techniques to help teachers, learners, and other stake- 2015), and systems that try to empower people to KROGHUH[VDUSORHXQGQGHUVWDUHOQGHYDXWVQWHUUDFHV make better-informed decisions. For instance, visual collected in various (online) environments. The overall analytics systems (Shneiderman & Bederson, 2003) objective is to improve (human) learning. HGFLVWLWQFHR[HKWWHKHUYRIRFOHYDULHZDYUSRLHG QRV The goal of this chapter is to provide a guide to practi- that can be made, and the potential implications of tioners and researchers who want to get started with those decisions. the development and evaluation of learning analytics GRWVRWWXSFPRHUUWHQVJKHPIWR'WLDRSOQW\DDLKVQJ HJ[VHUHYUSRYDDXPDOLHGLGGQDHGFQHDVKE:RDUGV - OSHVKXWQHTFLYXLVDQOWRLL]KDZLHOFUKLFXQJQUXEHPQ \D ples, to address the following items: to the remarkable perceptual abilities that humans 1. :KWNDLIGQGRWDDFDQEHYLVXDOL]H"G possess. The difference between the two approaches is like the difference between a self-driving car and 2.  HODUUHQHGGQWHLQYXLVDOQVWLR]WLDHKDHUPRKZUR) DFDUZLWKDKXPDQGULHYU'DWDPLQLQJXVHVDXWR - W"PRU HVKHDUHWQDHDDFUFKHKHUHUUJ matic pattern matching for remote control while the 3. :\KZKWLDVWRKHJDIWORKHYLVXDOL"]WDQLR dashboard provides visual communication to assist a 4. QFRWUDLWHLQ:KFKLYXDVL"OLGH]EHWGDDHWKFDQZR+ human driver who remains in control of the vehicle. :WHSSODWFKOLRH"LFGQEROKEUVHWLTDDDXHVQULHV There is a certain philosophical or ethical side to this data formats, et cetera can be used for the technical notion of two approaches as well: if learners are always FDQUHFSLHDGQUNRZ:ZRĂWQLHPHOKSWPW"DQDRLV W\WLHSKSROHHGYWHK\FDQZRK[HQWRGWRZKWDWGOR FDO EHXVHGWHSWORRGHYKHYLVXDOL"]WDQLR 21st-century skills of collaboration, communication, In addition to these four questions, we elaborate on FUIWLUXPRHLFWQGDKDDLOQWFDNUHDWDLQGLQYJWL2U\" - evaluation aspects that assess the usefulness and FWHLTXLESSL]HHHQHFPGRWVFHK\DQZRKHWOOHHQPYDO potential impact of the approach with the knowledge, skills, and attitudes to participate PHWKRGQRRVIFXVHZIVRWFXKFLKLWSHWDOQ,HULQVO\\" that augment the human intellect, through visual approaches for learning analytics (Engelbart, 1995). PG 143 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS SSODLFWDQLRVWROHDUQVFSDHVQRODUHJXSEOLFGLVSO\DVÝ Information Visualization S 'DVKERDUGVWKH\VD\ÜW\SLFDOO\FDSWXUH HFWYUDLWHLQIRHXVHWKVLYQXDVLRWOLL]DQURWRLPDQI, YXDVLO DYQGLVXDOLW]OHHUIDRDFHUVQLDFQWJLYWLWLHRUGHRULVQ representations to amplify cognition (Card, Mackinlay, DGQPDNLQHVQJHVDGQQHFWRĂLHUHVVQDZHUDWHRPRUS & Shneiderman, 1999). It typically focuses on abstract HQWOHRDEOHDURGHJQāHDUWRQDHWVOVQSUGURJDFNUHVV VWU'DDZLGJWWKDURWLIWUXKRHDUSHLWVZQHQDWUDGQLR XVHPDHVNSUSDH7HK  SRDJÝOVWHKVHZWDRGUV IXO V9SDFH'LVXDDUROQDO\WWXSVLSFHVVFKSHPLDāFVLV distinction between various types of dashboards: QRXELOGLQJPRGHOVDQGYLVXDOL]LQJWKHVHLQUGHRUWR XHUVYIH\X$OPRGHOVWKHUEHXāHWQQHGHUWVWURHUDQG 1. IFHDWWRUSSUVXIGRFDHDWLQRLWW'DDKVKEWORDDGUV RJDOIRLUQRIPWDQLRYLVXDOL]WDQLRLVWRUHO\QRXKPDQ lectures, so as to enable the teacher to adapt the perceptual abilities for pattern discovery (trends, gaps, teaching, or to engage students during lecture outliers, clusters). These patterns often become more sessions. apparent visually than numerically. As Ware (2004) 2. NURZJSXRUHFDIWRHFIDUSRSXVWWWKDED'DKRVGVU H[SODLQWVL and classroom orchestration, for instance by The human visual system is a pattern seeker of YLVXDOL]LQJDFWLYWLLHVIREWRKLQGLYXLGDOOHDUQHUV enormous power and subtlety. The eye and the and groups of learners. HDDUOOS\OHYLVVDPDUPRIQDLUEHWKIR[UHWRFDXOYVL O 3. HODUQHGGQHOEURQROLHQUSSRXVWWKWD'DEKVRDGUV - processor that provides the highest-bandwidth LQJDQHDUO\IDPRXVH[DPSOHLV&RXUVH6LJQDOV channel into human cognitive centers. At higher WKDWYLVXDOL]HVSUHGLFWHGOHDUQLQJRXWFRPHVDV levels of processing, perception and cognition DWUDIāFOLJWKEDVHGQRJUDGHVLQWKHFRXUVHVR are closely interrelated, which is the reason far, time on task and past performance (Arnold ZK\WKHRZUGVÜXQGHUVWDQGLQJÝDQGVÜHHLQJÝ 3LVWLOOL DUHV\\QQRPRXV [SY L 0RUHVRSKLVWLFDWHGDQGFRPSOH[YLVXDOL]DWLRQV $VVXFKYLVXDOL]DWLRQKDVWKHSRWHQWLDOWREHPRUH for detailed analysis of course activity by teach- precise and revealing than conventional statistical ers are the focus of the Student Activity Meter computations (Tufte, 2001). * YRDHUHUEHUW916$'VW3X DY3D3 UGRO 6WWDLFYLVXDOL]WDQLRVLH DQL PWDHJ\SLFDOO\YSURLGH RIFXVHVQRWKHYLVXDOL]WDLQRIRVRFLDODFWLYWL\IR answers to a limited number of questions that a user OHDUQHUV %DNKDULD 'ZDVQR PLJWKKHYDDERWXDGWDDVHWUR)H[DSPOHVRFDOOHG In terms of what is being tracked, the possibilities infographics are often used for storytelling in jour- WFQRLXHQWRH[SDQGDVQHZQROLQHWUHUDFNVEHFPHR YLVRXHFYDWHDLYOLD]WQDWDRONRLQJUHHYZRQ+DOLVP QRL available, capturing more detail of what learners and often leads to new questions that can only be answered teachers do. As well, new sensors proliferate that by interacting with the data itself (Few, 2009). Adding can likewise capture what people do in the analog G\QDPLFLWQHUDFWQLRWHFKQLTXHVWRWKHYLVXDOL]WDQLR world. This second data source is evolving especially therefore, is often necessary to design meaningful rapidly, with mobile devices that now include sensors YLVXDOL]WDLRQWRROVWKWDHQFRXUDJHH[SORUWDRU\GWDD to report physiological, emotional, and other kinds of analysis. learner characteristics that have so far mostly eluded $QRWKHUDGYDQWDJHRIYLVXDOL]DWLRQLVWKHDELOLW\W R automated capturing. Besides tracking, self-reporting reveal problems with the data itself; for instance, about can also be a valuable source of data. Although more the way the data has been collected. Especially in the HUSURUQRHUDQGGLIāFXOWWRVXVWDLQV\VWHPDWLFDOO\ case of learning analytics, where (semi-) automated self-reporting offers an opportunity for awareness, trackers often capture traces of learner activities, this UHĂHFWDQLRDQQGVDHOIO\VLV advantage is valuable for quality control. As for what can be incorporated into a dashboard, HUEHU9WHWD OOLV WVWROKHIZORLQJNLQGIGVRWDD WHAT FOR, WHOM, WHY, HOW? 1. Artefacts produced by learners, including blog posts, shared documents, software, and other :KDWIROORZVLVDQRQH[KDXVWLYHRYHUYLHZLWLV artefacts that would often end up in a student LPSRUWDQWWRUHFRJQL]HWKHYDULHW\RIDSSURDFKHV project portfolio. This variety is not surprising given the wide variety 2. Social interaction, including speech in face-to-face IROHDUQLQJDQDO\WLFVGWDDWKWDFDQEHYLVXDOL]HGURI group work, blog comments, Twitter or discussion a wide variety of audiences and reasons, in a wide forum interactions. variety of ways. 3. Resource use can include consultation of documents HDOUQLQJIRXVU\HYDWQHHVUS  DOWHUUEHHW9 DQD\OW - (manuals, web pages, slides), views of videos, et LERPPVDOOUÜIDPRQJULOQFWQVSLJRSDLDGEDKDRVGUFLV HO PG 144 HANDBOOK OF LEARNING ANALYTICS Figure 12.1.RUDJHUZIYYLHVEDGJHRHQSHULRGLYWÚHUZ$VWXGHQHUVRQDO%DGJH2YYLHGØ3YL%DGJHERDUD 1RS 7 (bottom) Navi Surface: students actively using the tabletop display application during a face-to-face session (Charleer et al., 2013). cetera. Techniques like software trackers and a node link diagramWRHQDEOHIXUWKHUH[USORWDQLRIR eye-tracking can provide detailed information O[SHFDQWVQHGWVXVWJKLQUHWKRJQ$RPVHJGDEHVHWK HUR DEWRXZKWDSDUWVIRUHVRXUFHVH[DFWO\DUHEHLQJ ZWKRVWLFKKHUHWXGHKQHDYVDUVSQHGHFLEDGDHāJFVV used and how. a means to compare and discuss learning progress. 4. Time spent can be useful for teachers to identify Figure 12.2 shows a dashboard that uses grades to students at risk and for students to compare their predict a student’s chances of failing a particular own efforts with those of their peers. FRXUVH2FKR DHUEHU9W&KLXOL]D 'XYDO EH - fore she starts. The dashboard is intended to support 5. Test and self-assessment results can provide an teachers in giving advice to students on their learning indication of learning progress. HVUHSGEDDRKGVUWHKSVHFLFāDO\OHUR0WMHFUUWDRHVL WVQ Figure 12.1 presents one of our more recent dashboards SDIDUWLKLWVVFWWLXQIHGRXODU  OLOHLNRKWRGHK DOLQJ 7.KH'OHU &N2XYKG DDU[O U/OXHLRHU]LROVD course in which she is interested. The dashboard uses dashboard tracks social data from blogs and Twitter. colour cues to indicate whether the risk of failure, G6FKXWDDFWDUHRJL]HDGV artefacts produced, is then based on past performance, is low (green), medium YLVXDUROIL]HG students. The goal is to support aware- WHPXRFRWHKQR'SHGQHLQJHU G KLJKUR OHZ\RO WHK ness about learning progress and to enable discussion teacher can advise the student to take the course or to in class. To support such awareness and discussion, HXUTHUSWDDāNUVWLQJDVKFXVDWUHOGLHVQWFVXDYLVV WLHLV social interactions of students are abstracted in the course. The dashboard also supports several interaction form of learning badges for students to earn. Students techniques that enable the teacher to indicate which H)DLJUHQGHKYDWHK\HGEDJVZKKFL[HHUROSWQHKFDQ - data should be taken into account to generate this XUH WRSWKURXJKWKHYLVXDOL]WDLRQRI icons and prediction, including sliders at the bottom that enable colour cues*UD\EDGHJVKHDYQRWH\WEHHQHDUQHG the teacher to specify the range of data in terms of 7KHEWRWPRSDUWIR)LJXUHVZKRVDYLVXDOL]WDQLR H\DUURH[V)DSOHPLIDVWWGXGHQLGSRURO\LQ%LRORJ\ developed for collaborative use on a tabletop that uses LQ*UDGHHURWNGXEZKDUGHUDQGGLHGOZOLQ*UDGH Figure 12.2. WDOFKRDH2RXUVH FLāFFIIDLOLQJDVSHHOLKRRGRWVWKHOLNHVHQHSUWUGWKDDGDVKERDUY0X PG 145 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS WKH*UPDGHDUNFDQEHGLVUHJDUGHG SFPRDUHGWWRRKHUVWWXGHQ"V • How much do I contribute to the discussion forum, HOW TO GET STARTED SFPRDUHGWWRRKHUVWWXGHQ"V In both cases, we deliberately only list questions that To leverage the advanced perceptual abilities of humans VWDUWZWLKÜÝZÝKÝÜWDDÜZKRZFKXKHQQGPÜZKRIR - HLJVGDWDSUWHQVUHGYLRVFDGQHU[HROSWPHKSOHKWR UHQ Ý7WHQKHVHVSHFLāFGLUHFWTXHVWQLRVFDQEHGLUHFWO\ must create a visual representation or encoding of the PDSSHGLQDGDWDVHW4XHVWLRQVOLHNÜ Why did this data (Card et al., 1999). Several steps, outlined below, student have to enroll twice in this course?DUHQVZWHKÝ can be distinguished in this design process. H[WUQSORWUWPRDHDUKR,LEQGWXDHQLLFPUVHW\\DDURV he did not spend enough time on the course material, Understand Your Goals did not interact with fellow students on the discussion PGRDLSUREOQHPWKHNHWJZQRWWRLLVQJVWāHSU7VWKH forum, started to study the course material too late, the data set, the intended end-users of the tool, the XRZODQUHVZWGRL$ITHXVWWRQQUHāFRLKQRXWOVRDGQ G typical tasks they should be able to perform, and so EHÜ Are students more eager to work on assignment on. The following questions need to be answered at 1 or assignment 2?ÝHQ(YLIFKXPGWDDLVFWSDXUHGWL this stage: GLLVIāFDWXTXHQRVVWOHUWZROLYYQLRQVXLKPQJPRDQ - 1. Why:WLRKWJDYVWIKHRDKHOLVXDO:L"]WDKQLRWD tivations based on a plurality of (un)known variables. TXHVWQLRVDEWWRXKHGWDDVKRXWDOGLQVHUZ" Especially in the early phase of design, it is therefore GLUHVSQFRHRWFIFLXāFVWIRDGHYRDDVWLLHUHQQGVDEOH 2. For whom"HGGQWHLQYXLVDQOWRLL]DWHKLVPRKZUR) questions. Are the people involved specialists in the domain, ULRQYLVXDOL"]WDQLR Acquire and (Pre-)Process Your Data Building a visual dashboard typically entails a data-gath- 3. What'R\GO"SDLVYXLVDOQLWWR]HLDKZLOWGODD:WKD HULQJDQGSUHSURFHVVLQJVWHS9LVXDOL]WDLQRH[SHUWV WKHVHGWDDH[KLELWDVSHFLāFLWQHUQDOVWUXFWXUH suggest that this step takes 80% of the time and effort OLHWNLPHUDQDKHRWLHUURNZD"\UFK (ODPT'YQGF V0Q RWHQHSOWRDLVHUVKHWUYOVXOV 4. HowRDJ"OYWLVXHZKSWSUVXRDHKLOWLOZ]WORQD+RL identify the following intermediary steps: How will people be able to interact with the vi- 1. Acquiring raw data: It is important to have a clear GHY"WRXLLWVFWXSXHQHQWGHDG:KHLOVL"K]WWDDQLR idea of where the data will come from (e.g., the log VWGDDHWKUWVHGGDLXJQQQGDQ[HDPLQLJQIFHDUXO\O\% WH āOHVIRWKH/06DVVHVWVPHQUHVXWOVWR KHUDQG a variety of questions about the data can be formed. when the data will be updated (continuously, not at Having these questions in mind can be useful when YDDLODEHGWDDW:HKLWLUHQOVSOHFL YDFāOVWDDOO HOE URG)DEKVRDWGUHKURIāGWWOUHDFTDXDLDLUGQQJLQJ [H - WKURXJKDQ$SSOLFWDLRQ3URJUDPPLQJ,WQHUIDFH ample, consider a data set that contains the following $ ,D3 QH[SURWāOHUVRWPHRRKHUV"RXUFH learner traces: 2. Analyzing raw dataHOFDHQGEHWRHQHGP\DW'DD • access to learning resources if some values are missing or er roneous, or • WLQSPHRDHLJQGLJWLDOWH[WERRNV pre-processed to compute aggregate values (mean, PLQLGPXWDPDDP,QQFD[HDW HULOHWD\XPVLPV • contributions to discussion fora distribution can also be an issue: are there apparent • time spent on assignments WRXOLHUVXFOVWHUVHWF"HWHUD )URPWKHVHWUDFHVZHFDQGHāQHVHYHUDOUHOHYDQW 3. 3UHSDULQJDQGāOWHULQJGDWD : Using the initial questions as a starting point in the design process. A questions from step 1, choose the relevant data teacher might ask questions like these: IPWURKHSRIDRORQDO\]HGUZGDWDD • When did students start looking at the course Mapping Design PWDHULD"O Important in the visual mapping design is to choose • What is the average time that a student spends a representation that best answers the questions you UHDGLQJWKHWH[WERRN" want users to be able to answer, i.e., that serve your YLVXDOL]WDLRQJRDORUIWKHLWQHQGHGWDUJHWDXGLHQFH • "WQJHLVVQPDVKLQRNURZUHWH3GGLXUVRK\PDQZR+ 7KHUHH[LVWVDXPWOWLXGHIRDWOHUQWDHLYV2QHZ\DWR • +RZRIWHQGLG3HWHUDVNDTXHVWLRQRQWKHGLV - start is to look at the measurement or scale of each FXVURVQILR"XP data characteristic. Nominal or qualitative scales differentiate objects based on discrete input domains, A student will probably ask similar questions: TXWRDFKHUOWDUHWDFLROURJVDWLVHDVFKHXVVLLYāFWDQLRV • How much time do I spend on an assignment, WRZKLFKWKH\EQHORJ4XDWQWLWDHLYVFDOHVKHYDFQR - PG 146 HANDBOOK OF LEARNING ANALYTICS WLXRXQVLWSXQGPRDLQVHJ @> 2UGLQDOVFD OHV between the numbers quite originally by relating have discrete input domains where the order of the them to age, where a person of 37 can easily lift WQWHHEHZHFQHGULIHI[HFWDWHKWXEWPUDWHVWVQHPHOH HK weights, while a person of 73 might already need YDXHOVGRHVWQR'HSHQGLQJQRWKHVFDOHIRWKHGWDD a walking stick. characteristic, one can choose how to encode this • )LJXUHGDGGXVPVFOHVL]H GDWDYLVXDOO\)LJXUHGHSLFWV0DFNLQOD\VÚ • FLFULGH]VXTHDO\ODQIRGKLDJVQHXVVHJ)LHXU HO ranking of visual properties to encode quantitative, with 75 versus 37 stripes. ordered, and categorical scales. For instance, the spatial position of an element is useful for encoding • Figure 12.4f: uses a position in a Cartesian coor- quantitative, ordered, and categorical differences. dinate system. This is why scatterplots have been used so often to • KVQRWLDOHUHORKZRWDUSWDXVDYHL]OVLJJL)HXU SL convey a variety of information. Length, on the other between the numbers. hand, can encode quantitative differences, but is of • Figure 12.4h: assumes a time-based relationship less value for encoding ordered and categorical dif- between both numbers, which leads to a negative ferences. Shape is at the bottom of the ranking for trend line. YLVXDOL]LQTXJDWQWLWDDHLUGHRYUHQGGGLIHUHQFHIVWXE is more often used to depict categorical data. • Figure 12.4i: uses point clouds. āG/HOLRZW\SURWRW\SHVVXFKDVSDSHUVHNWFKHVDUH • )LJXUHMYLVXDOL]HVDQXQEDODQFHGVFDOHWR often helpful during the design-mapping step. Figure represent a difference in weight. GHSLFWVDQHH[UFLVHJHLQYWRWKHSDUWLFSLDWQV IR • āZJWLWXKHKUHIRVL]H)WLUJFRHKOUHWDHXNVUH 9/W'2LDVXHZDDDXRUOQU$QLQÜ%<UQDJLOW\QHJKWLFV Ý WKHVL]IWHRXKHQPEHU W/WXUHRWDURKHJLDUDQWD$LQO]LHQQGJDO\6XWPLFPHUV HHVDUUGHGXOFLQD3WUDVSQLFWL /$, 6WWHWQV,LX UVHKF RUSW\LOHGāKJLKJQLGRFQHDOXVYLDJQLWFHOHVU$IHW WV\HSRW with good knowledge in learning analytics, but limited XEXYFVLLEHDQWOJLQVXDOLW ]RWURDDRO7EOHQLVORXDLHN WRWGHNDVHUHZ\7HKQYXRWDVLLOL]DWXREDHJGHOZNRQ HDN UOELYXLVDQ[OWHRLWL]LVDLQJUR OHF([IRRV0FULWQHYH DUHVL PLWXQHVWRHVWNFKDOOSRVVLEOHZD\VWRYLVXDOL]H D OLH3NURFHVVL QU'JRMV VLGSOHPWDWIXVRQDHWRPEZ7HU^HHUF[VKHL`VH Clearly some alternatives work better than others, illustrated to participants that from the moment they GHSHQGLQWQRJWFQKHRH[WXDOHLLZJHDW]KJWDQG QLR GLIEUWQRVDWWāFLLRVXDQHVWWVWUOYNWFKLURLLWPQVXJDO age) and the ability to be interpreted by users (e.g., the HQFRGLQJVIRGWDD7KLVLVUHĂHFWHGLQWXKHQPEHUIR mental model of a balanced scale). There is, therefore, sketches that two teams of two persons each were able QREHVWZD\WRYLVXDOL]HDGDWDVHWEXWVRPHWHFK - to generate in 15 minutes (see Figure 12.4a and 12.4b). niques have been proven to work better than others, By sketching, more ideas and questions about the data UHR[IDSOHP set are often raised, which in turn leads to new ideas • 3LHFKDUWVDUHXVXDO O\DEDGLGHHZD ) UYRILVXDOL]WUHDR[QLR)DSOHP • Bar charts can be quite powerful. • Figure 12.4c: participants represented the difference • &RUGRLQWDHGJUKSDVHQDEOHULFKH[USORWDQLR Figure 12.3.GHGHUHRUWLYWDWLWHULVWLFVRQTXDQDFWDFKDURSHURUGDIYLVXDOSUWLHVIDQNLQJR UV \Ú0DFNLQOD and categorical scales. PG 147 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS • J'UKSDLFIRVHWD\YFWQQRRGRHQDGG\QWLQLRDO • LVPLODUWHVLLāGQWRWDUSWHQVDGQHDYVXODSUP&RLQJ information and force the reader to deal with and differences. UHGXQGDQWDQGH[WUDQHRXVFXHV /HY\=DFNV • Sorting items based on a variety of data values 7HUY 6VNFKLD\QR or metrics. • Scatterplots and parallel coordinates are good • Filtering values that satisfy a set of conditions. representations for depicting correlations. In • WXRWVGDQHVXDYOFLāFHSVHPDNRWWGDDWLJJKQL+JOKL addition, Harrison, Yang, Franconeri, & Chang visually without making all other data disappear, (2014) found that among the stacked chart variants, DVLVWKHFDVHZWLKāWOHULQJGWDD WVWKHHEDDFNGVLUJQLāFDWQWRXO\SHUURIPHEWGRK WKHVWDFNHGDUHDDQGVWDFNHGOLQH(OOLRW • Clustering or grouping similar items together; for has presented a nice overview of these studies. H[DPSOHE\DJJUHJWDLQJTXDWQLWWDLHYGWDDHJ average, count, et cetera) to view it in a higher Documentation or lower level of detail. $VZLWKDQ\GHVLJQH[HUFLVHLWLVLPSRUWDQWWREH H[SOLFWDLEWRX • $QWQRWDLQJāQGLQJVDQGWKRXJWKV 1. Rationale: W hy were certain decisions made, • %RRNPWDVSUHKHUHFYUNGRFQRDLULRLāLFHQZQJJ ZKWZDDVWKHLWWQ"HQ data to enable effective navigation. 2. Alternatives: Which alternatives were considered Heer and Shneiderman (2012) is essential reading on DHUHWQGZ\ZKKHWZ\QRWLK"KHOG interactive dynamics for visual analytics. The authors QFRG\WLWQUHQKFDWDWDHPLYFLVIRW\DPRQR[DHUWSVQH - 3. Evolution: How has the design evolved from early tribute to successful visual analytic tools. For each HVWNFKHVWRDIXEOZROOQLSPOHWPHQ"WDLQR:KWD WDVNFWDUHRYJDU\LRXVH[LVWLQJYLVXDOL]WDQLRV\VWHPV URIZKWDDGQHUDQVVWRSHFXQRFDOURIRPGLHāGZDV are described with useful interaction techniques that implementation or other reasons (logistics, lack support the task at hand, such as brushing and linking, IWRLPHWRKHUUH"D VQRV KLVWRJUDPVOLGHUV]RRPDEOHPDSVG\QDPLFTXHU\ Add Interaction Techniques āWOHUZLGHWJVVPDOXOPWOSOHLGLVSO\DVURWUHOOLVWSORV 9LVXDODQDO\VLVW\SLFDOO\SURJUHVVHVLQDQLWHUDWLY H multiple coordinated views, visual analysis histories, UHāQHPHQDHG[QURWOFSDUHQWRLYDQRLHLZIRURSFHVV W and so on. H+HU 6KQHLGHUPDQ %URHIHDQDO\]LQJZKLFK Evaluate Continuously LWQHUDFWQLRWHFKQLTXHVDUHXVIHXUORDIVSHFLāFYLVX - FUQRFIRQWRUOLDEHDRWHKRHFVUVSLJHVGQWHK'XULQJ WHH DOL]DWLRQDSSOLFDWLRQLWLVXVHIXOWRXQGHUVWDQGW KH personas and scenarios can be very rewarding as it typical analytical tasks performed by teachers who helps to focus the design, development, and evaluation want to understand how their students are doing in HUHYDVL\\VYUWHOH,YWDLLWKHVQXVZIDRKWDOQLR]WDQLR FODVV6HHUYDOWDVNWDQR[RPLHVKHDYEHHQGHVFULEHG ODGQF\DGQÝH\HÜKFXPWRRZWKL\DZDFDUUHGLWHJWR HRV in literature for this purpose. Common tasks include: YLXVDWHOKLW]D\ZKDGQPRZKURIZKWDWHKIRWUNFD QRL Figure 12.4. Sketches of a small data set of two numbers {75, 37}. PG 148 HANDBOOK OF LEARNING ANALYTICS LVEHLQJGHVLJQHG*HQHUDOO\DXFHWVQHUUHGGHVLJQ CONCLUSION WQHPSHORHGYWLZHUUSRWDFHHWLLHYGKVUSSRDKFD &'8 that keeps the target users in the loop in continuous JRORHWRLGHKPDGQWSVHFQRFYXLVDQOWLRL]DUQRWRPLQ,DI V FFOH\VIRGHVLLJWSOHPHPQ×W×HQDYDQLRXOWDQLR,QWKLV are key enablers for way, the development can focus on the most relevant • Learners to gain insight into their learning actions issues for teachers or learners at all times. and the effects these have. 7KHHYDXOWDLQRIRLQURIPWDLQRYLVXDOL]WDLQRV\VWHPV • Teachers to stay aware of the subtle interactions is essential. A plethora of techniques can be used, in their courses. LQFOXGLQJFWRQUROOHGH[SHULPHWQVWKWDHYDOXWDHGLI - IHUHQWYLVXDOL]DWLRQDQGLQWHUDFWLRQWHFKQLTXHVRU • Researchers to discover patterns in large data āHVOGWXGLHWVKDWDVVHVWVLKHSPDFYDIRWLVXDOL]WDQLR sets of user traces and to communicate these QROHDUQLQJ O3DLVDWQ 7KHOWDWHUWDHNSODFHLQ data to their peers. natural environments (classrooms) but are often time YXQLVLXWDHKKODLVKF]WSWWDUHDQLRKLLVQZVRK$QV THX FRQVXPLQJDQGGLIāFXOWWRUHSOLFWDHDQGHJQHUDOL]H potential to help shape the learning process and 1DJHOHWDO HUE9HUWHWDO  VXJJHVW WKH HQFRXUDHJUHĂHFWLRQRQLWVSURJUHVVDQGLPSDFWE\ following evaluation techniques: creating learning analytics dashboards that give a 1. Effectiveness, which can refer to engagement, concise overview of relevant metrics in an actionable higher grades or post-test results, higher reten- Z\DDQGWKWVSSXDURWWKHH[USORWDISWQRLRDWHUQV tion rates, improved self-assessment, and overall 'HVLJQLQJDQGFUHWDLQJDQHIHIFWLHYLQRUIPWDLRQYL - course satisfaction. VXDOL]WDVQLR\OHVURWIDHPUQLQDJQDO\WLFLVDVDQUW DV 2. HDOUHQURUHKFWHDDIRWHLPIRHXVHWKLQFQI\(HLFā U HODUQLQQ[HRSUHPRDGWLLQVHWERKHHQGVHLGVJUHQWHK J theories and paradigms as well as techniques rang- 3. Usability and usefulness evaluations often focus ing from visual design to algorithm design (Nagel, on teachers being able to identify learners at risk 6SHQFH ,QWKLVFKDSWHHUZKHDYEULHĂ \ or asking learners how well they think they are YLVXDDOGHLYVLLDQJV]WUWWHDSLQVLRWQKHQLXRURVFXHGG performing in a course. process, from raw data analysis to effective dashboards Typical evaluation instruments include questionnaires evaluated by target users. UHWURUWWRDNVLHPHUHZ[KHSUHWWORQLUQRFHHVOPGUR V PWOHDGHWLRDPHUFHWDHHWUHQYDUDHXOW DH'GLOOHQ - bourg et al., 2011). REFERENCES WDVHVWXGHQH\DUQLQJDQDOWLFVWRLQFUVLQJOHGXH8W3XULJQDOVD  &RXUVH6$UQROG.( 3LVWLOOL0' nd success. Proceedings of the 2 International Conference on Learning Analytics and Knowledge (LAK ’12), 29 &0N$RUZ<H 1×%&&DQDGD SSHUYRXDQF\90D$SULO× WLRQDFWHUWLQDOSDUIWHPSRUWLFLSDQZRHYLH\VHGÚ3$ELUZVRQ6  61$3D%DNKDULD$ ' Proceedings of st the 1 International Conference on Learning Analytics and Knowledge /FK0DU$.Ú )HEUXDU\× &0N$RUZ<H 1%DQII$%&DQDGD SS× QWLRQ,DGXFGHHEEDVHWWXWRULQJV\VWHPVWRZWHOOLJHQRPLQGLD)U\SHUPHHK $GDSWLY 3YVN\UXVLOR% th /HKQ (GV DQDVVRQ.9&)UXWKLHU*D* Proceedings of the 5 International Conference on Intelligent Tutoring SystemsGRLSULQJHU 6×DO4&&DQDGD SSHWURQ0-XQH × ,76 6KQHLGHUPDQ% (GV  -\G6.0DFNLQOD&DU Readings in information visualization: Using vision to thinkXIPDQQ3XEOLVKHUVJDQ.DRUOLQJWRQ0$0XU% RXJWLRQWKUFHĂHHQHVVDQGUKDUZYLQJDRPSU ,DO( Y 'XROD6/XLV-GULR]2N[-6.OHUHUOH&KDU DQ/3DPPHU3H9RRURJVWLH$0FLN%.UYDQ0.UIEDGJHV,WLRQVRHYLVXDOL]DWLYDFWHUHLQWLYDROODERUF - rd 8OOPDQQ (GV 'W 7GHLQKDU5QHVH03ULOOD: Proceedings of the 3 Workshop on Awareness and Re- QKDQFHG/HDUQLQJHFKQRORJ\(ĂHFWLRQLQ7 DSKRV&\SUXV SSSS×  6HSWHPEHU37(/Ú$5 W6 .DSODQ)  &XHQGHG4'ROHQK6%RQQDUHUPDQQ3YL+-$OD*\H=XIHUIJ3'LOOHQERXU th Classroom orchestration: The third circle of usability. Proceedings of the 9 International Conference PG 149 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS on Computer-Supported Collaborative LearningRO,SSYRQJ&KLQD RQJ.\+&6&/ ×-XO HVDUQLQJ6FLHQFIWKH/HW\RWLRQDO6RFLHWHUQDQ× , GLPH W9LVKWSVW2SHQGDWHHVHQHSWLRQLQPLQXWHV3UF VWXGLHVDERXWKXPDQSHU(OOLRW.  - PLQXWHVIHDUV]FKLHSWLRQLQFVWXGLHVDERXWKXPDQSHUHQQHOOLRW#NWRPXPF H,4WLYFROOHWDQGERRVWLQJRXUFFWHOOHWLQJWKHKXPDQLQXJPHQGDDUZR(QJHO  7EDUW' Communications of the ACM, 38 × Few, S. (2009). Now you see it: Simple visualization techniques for quantitative analysis. Berkeley, CA: Analytics HVV3U FHĂHUHQHVVDQGVHOIDUZRUDWHUIW\PHWLYLWDFGR$  7KHVWXGHQDUDO( 3HUEHUYW.'XDHUWV69Y*R - tion. CHI Conference on Human Factors in Computing Systems: Extended Abstracts\0D&+,($Ú × &0N$RUZ<H$XVWLQ7 1;86$ SS× VHEHUÚWLRQXVLQJ:HODRUUIFWLRQVR 5DQNLQJYLVXDOL]DRQHUL6 &KDQJ5 DQFDQJ))UDUULVRQ/<+ law. IEEE Transactions on Visualization and Computer Graphics, 20  × \VLVRUYLVXDODQDO\QDPLFVIHGWLYDFWHUQ , 6KQHLGHUPDQ% )HEUXDU\-HUH+ Queue – Microprocessors, 10(2), 30. FHVLQSHUVSHHQFHUDSKLFV"3XWHIWRXVJUWXLWLQJSUD $SULO *U% 6FKLDQR'HUVN\7Y(=DFNV-Y\/H - tive. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems$SULO ×&+,Ú &0N$RUZ<H 1×%&&DQDGD SSHUYRXDQF9 WLRQRUPDWLRQDOLQIHODIUWLRQVRWDHVHQDOSUDSKLFIJUWLQJWKHGHVLJQR $XWRPD -\0DFNLQOD Transactions on Graphics, 5  × WLRQ$PRGHODQGLPSOHWLRQYLVXDOL]DRUPDVLQLQIGVXWLOL]LQJ*38DUZR 7 YLVW1'RQQHO% (OPTF0 - mentation of image-space operations. IEEE Transactions on Visualization and Computer Graphics, 15 DQVSRURXFKLQJWUW 7WFNO. 5DWL& .ORHN[-H$.OHURHUDQGH0DO(9YWDQ0'X0DLDJHO71 th WRSVHWDEOHWLYDFWHUWRQLQDQVLWDQSXEOLFWURSROLWU\RQYLVXDOL]LQJPHDVHVWXG$F Proceedings of the 12 International Working Conference on Advanced Visual Interfaces\ SSWDO\&RPR,0D× 9,$ &0N$RUZ<H 1× Nagel, T. (2015). Unfolding data: Software and design approaches to support casual exploration of tempo-spatial data on interactive tabletops. Leuven, Belgium: KU Leuven, Faculty of Engineering Science. NLQRU\DUQLQJDQDOWLFV:WLRQLQOHWHU\YLVXDOL]D 8QFWDLQDO( HUEHUYW.&KLOXL]D. 'XFKRD;92 progress for IEEE Transactions on Learning Technologies. nd WLRQDOXDYWLRQHWLRQYLVXDOL]DRUPDILQI 7KHFKDOOHQJHRW& ODLVDQ3 Proceedings of the 2 International Working Conference on Advanced Visual Interfaces \ SS×WDO\*DOOLSROL,0D ×9,Ú$ New York: ACM. 00LKDHVFX&/XQD-RV3WURQ$0LHUFHU00\FKHQL]NLHR&3RPHU5*DULR-&%RWLFWRV26DQ th DLV0 (GV   D6 'HVPDUWXUHQW]$9YLRHUVKN+HQR3RU0 Proceedings of the 8 International Conference on Educational Data MiningDWLRQDO(GXFWHUQDQSDLQ,-XQH0DGULG60 × (' - W\WD0LQLQJ6RFLHDWLRQDO' Shneiderman, B., & Bederson, B. B. (2003). HĂHFWLRQVUDQGHDGLQJVRUPDYLVXDOL]DWLRQ5WLRQLQIRIDIWFU7KH . Burl- XIPDQQ3XEOLVKHUVJDQ.DRULQJWRQ0$0 st Spence, R. (2001). Information visualization: Design for interaction, 1 ed. Salt Lake City, UT: Addison-Wesley. nd Tufte, E. R. (2001). The visual display of quantitative information, 2 &HVV//DSKLFV3UH&7*UG&KHVKLUH GV$QDUQLQJGDVKERDU /H N[- .OHUD*DUU$VVFKH)3WRV-DO(6DQDHUYHUEHUWV6'XYW.*R9 overview and future research opportunities. Personal and Ubiquitous Computing, 18 × nd Ware, C. (2004). Information visualization: Perception for design, 2 ed. Burlington, MA: Morgan Kaufmann 3XEOLVKHUV PG 150 HANDBOOK OF LEARNING ANALYTICS http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Handbook of Learning Analytics Unpaywall

Learning Analytics Dashboards

Loading next page...
 
/lp/unpaywall/learning-analytics-dashboards-S00Csv4sle

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Unpaywall
DOI
10.18608/hla17.012
Publisher site
See Article on Publisher Site

Abstract

Chapter 12: Learning Analytics Dashboards HUEHUDOWDQG(ULN'XY-RULV.OHUN[.DWULHQ9 Department of Computer Science, KU Leuven, Belgium DOI: 10.18608/hla17.012 ABSTRACT 7KFKLVWSSUDHHUVWHQOHVDUQLDQJQDO\WLFGVDVKERDUGWVKYWDL VXDOLOH]HDUQLWQJUDFHWJVRHLY HODU(WURDS[FHHKQVDLVLWHOSQPQRVJLQXVLJVWUHKV GDWVKEGHRKDWVDHGUVDZKWDIRZRVKQUH LVDRJOWHKWKZDHGGQWHLQDHUWH\KPRKZURIXVH QWGRGLLDYQ,XLVDOL]HEGHFDQWGDDZRKDGQ guidelines on how to get started with the development of learning analytics dashboards are presented for practitioners and researchers. Keywords:,UQRIPWDQYLRLVXDOL]WDQLROHDUQLQJDQDO\WLFVGDVKERDUGV In recent years, many learning analytics dashboards BACKGROUND have been deployed to support insight into learning data. The objectives of these dashboards include To Augment the Human Intellect providing feedback on learning activities, supporting There is a strong contrast between intelligent systems QHPHDJJQHLJHQDVUFLQPDNLJQQRLVLFHGGDQQRFHWLĂHU W that try to make decisions on behalf of people, such and motivation, and reducing dropout. These learning as intelligent tutoring systems (Brusilovsky, 2000) DQDO\LGUQRWDYIVKLFSESODP\VRWDLDVXQLURGDVOL]WDQLR and educational data mining systems (Santos et al., techniques to help teachers, learners, and other stake- 2015), and systems that try to empower people to KROGHUH[VDUSORHXQGQGHUVWDUHOQGHYDXWVQWHUUDFHV make better-informed decisions. For instance, visual collected in various (online) environments. The overall analytics systems (Shneiderman & Bederson, 2003) objective is to improve (human) learning. HGFLVWLWQFHR[HKWWHKHUYRIRFOHYDULHZDYUSRLHG QRV The goal of this chapter is to provide a guide to practi- that can be made, and the potential implications of tioners and researchers who want to get started with those decisions. the development and evaluation of learning analytics GRWVRWWXSFPRHUUWHQVJKHPIWR'WLDRSOQW\DDLKVQJ HJ[VHUHYUSRYDDXPDOLHGLGGQDHGFQHDVKE:RDUGV - OSHVKXWQHTFLYXLVDQOWRLL]KDZLHOFUKLFXQJQUXEHPQ \D ples, to address the following items: to the remarkable perceptual abilities that humans 1. :KWNDLIGQGRWDDFDQEHYLVXDOL]H"G possess. The difference between the two approaches is like the difference between a self-driving car and 2.  HODUUHQHGGQWHLQYXLVDOQVWLR]WLDHKDHUPRKZUR) DFDUZLWKDKXPDQGULHYU'DWDPLQLQJXVHVDXWR - W"PRU HVKHDUHWQDHDDFUFKHKHUHUUJ matic pattern matching for remote control while the 3. :\KZKWLDVWRKHJDIWORKHYLVXDOL"]WDQLR dashboard provides visual communication to assist a 4. QFRWUDLWHLQ:KFKLYXDVL"OLGH]EHWGDDHWKFDQZR+ human driver who remains in control of the vehicle. :WHSSODWFKOLRH"LFGQEROKEUVHWLTDDDXHVQULHV There is a certain philosophical or ethical side to this data formats, et cetera can be used for the technical notion of two approaches as well: if learners are always FDQUHFSLHDGQUNRZ:ZRĂWQLHPHOKSWPW"DQDRLV W\WLHSKSROHHGYWHK\FDQZRK[HQWRGWRZKWDWGOR FDO EHXVHGWHSWORRGHYKHYLVXDOL"]WDQLR 21st-century skills of collaboration, communication, In addition to these four questions, we elaborate on FUIWLUXPRHLFWQGDKDDLOQWFDNUHDWDLQGLQYJWL2U\" - evaluation aspects that assess the usefulness and FWHLTXLESSL]HHHQHFPGRWVFHK\DQZRKHWOOHHQPYDO potential impact of the approach with the knowledge, skills, and attitudes to participate PHWKRGQRRVIFXVHZIVRWFXKFLKLWSHWDOQ,HULQVO\\" that augment the human intellect, through visual approaches for learning analytics (Engelbart, 1995). PG 143 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS SSODLFWDQLRVWROHDUQVFSDHVQRODUHJXSEOLFGLVSO\DVÝ Information Visualization S 'DVKERDUGVWKH\VD\ÜW\SLFDOO\FDSWXUH HFWYUDLWHLQIRHXVHWKVLYQXDVLRWOLL]DQURWRLPDQI, YXDVLO DYQGLVXDOLW]OHHUIDRDFHUVQLDFQWJLYWLWLHRUGHRULVQ representations to amplify cognition (Card, Mackinlay, DGQPDNLQHVQJHVDGQQHFWRĂLHUHVVQDZHUDWHRPRUS & Shneiderman, 1999). It typically focuses on abstract HQWOHRDEOHDURGHJQāHDUWRQDHWVOVQSUGURJDFNUHVV VWU'DDZLGJWWKDURWLIWUXKRHDUSHLWVZQHQDWUDGQLR XVHPDHVNSUSDH7HK  SRDJÝOVWHKVHZWDRGUV IXO V9SDFH'LVXDDUROQDO\WWXSVLSFHVVFKSHPLDāFVLV distinction between various types of dashboards: QRXELOGLQJPRGHOVDQGYLVXDOL]LQJWKHVHLQUGHRUWR XHUVYIH\X$OPRGHOVWKHUEHXāHWQQHGHUWVWURHUDQG 1. IFHDWWRUSSUVXIGRFDHDWLQRLWW'DDKVKEWORDDGUV RJDOIRLUQRIPWDQLRYLVXDOL]WDQLRLVWRUHO\QRXKPDQ lectures, so as to enable the teacher to adapt the perceptual abilities for pattern discovery (trends, gaps, teaching, or to engage students during lecture outliers, clusters). These patterns often become more sessions. apparent visually than numerically. As Ware (2004) 2. NURZJSXRUHFDIWRHFIDUSRSXVWWWKDED'DKRVGVU H[SODLQWVL and classroom orchestration, for instance by The human visual system is a pattern seeker of YLVXDOL]LQJDFWLYWLLHVIREWRKLQGLYXLGDOOHDUQHUV enormous power and subtlety. The eye and the and groups of learners. HDDUOOS\OHYLVVDPDUPRIQDLUEHWKIR[UHWRFDXOYVL O 3. HODUQHGGQHOEURQROLHQUSSRXVWWKWD'DEKVRDGUV - processor that provides the highest-bandwidth LQJDQHDUO\IDPRXVH[DPSOHLV&RXUVH6LJQDOV channel into human cognitive centers. At higher WKDWYLVXDOL]HVSUHGLFWHGOHDUQLQJRXWFRPHVDV levels of processing, perception and cognition DWUDIāFOLJWKEDVHGQRJUDGHVLQWKHFRXUVHVR are closely interrelated, which is the reason far, time on task and past performance (Arnold ZK\WKHRZUGVÜXQGHUVWDQGLQJÝDQGVÜHHLQJÝ 3LVWLOOL DUHV\\QQRPRXV [SY L 0RUHVRSKLVWLFDWHGDQGFRPSOH[YLVXDOL]DWLRQV $VVXFKYLVXDOL]DWLRQKDVWKHSRWHQWLDOWREHPRUH for detailed analysis of course activity by teach- precise and revealing than conventional statistical ers are the focus of the Student Activity Meter computations (Tufte, 2001). * YRDHUHUEHUW916$'VW3X DY3D3 UGRO 6WWDLFYLVXDOL]WDQLRVLH DQL PWDHJ\SLFDOO\YSURLGH RIFXVHVQRWKHYLVXDOL]WDLQRIRVRFLDODFWLYWL\IR answers to a limited number of questions that a user OHDUQHUV %DNKDULD 'ZDVQR PLJWKKHYDDERWXDGWDDVHWUR)H[DSPOHVRFDOOHG In terms of what is being tracked, the possibilities infographics are often used for storytelling in jour- WFQRLXHQWRH[SDQGDVQHZQROLQHWUHUDFNVEHFPHR YLVRXHFYDWHDLYOLD]WQDWDRONRLQJUHHYZRQ+DOLVP QRL available, capturing more detail of what learners and often leads to new questions that can only be answered teachers do. As well, new sensors proliferate that by interacting with the data itself (Few, 2009). Adding can likewise capture what people do in the analog G\QDPLFLWQHUDFWQLRWHFKQLTXHVWRWKHYLVXDOL]WDQLR world. This second data source is evolving especially therefore, is often necessary to design meaningful rapidly, with mobile devices that now include sensors YLVXDOL]WDLRQWRROVWKWDHQFRXUDJHH[SORUWDRU\GWDD to report physiological, emotional, and other kinds of analysis. learner characteristics that have so far mostly eluded $QRWKHUDGYDQWDJHRIYLVXDOL]DWLRQLVWKHDELOLW\W R automated capturing. Besides tracking, self-reporting reveal problems with the data itself; for instance, about can also be a valuable source of data. Although more the way the data has been collected. Especially in the HUSURUQRHUDQGGLIāFXOWWRVXVWDLQV\VWHPDWLFDOO\ case of learning analytics, where (semi-) automated self-reporting offers an opportunity for awareness, trackers often capture traces of learner activities, this UHĂHFWDQLRDQQGVDHOIO\VLV advantage is valuable for quality control. As for what can be incorporated into a dashboard, HUEHU9WHWD OOLV WVWROKHIZORLQJNLQGIGVRWDD WHAT FOR, WHOM, WHY, HOW? 1. Artefacts produced by learners, including blog posts, shared documents, software, and other :KDWIROORZVLVDQRQH[KDXVWLYHRYHUYLHZLWLV artefacts that would often end up in a student LPSRUWDQWWRUHFRJQL]HWKHYDULHW\RIDSSURDFKHV project portfolio. This variety is not surprising given the wide variety 2. Social interaction, including speech in face-to-face IROHDUQLQJDQDO\WLFVGWDDWKWDFDQEHYLVXDOL]HGURI group work, blog comments, Twitter or discussion a wide variety of audiences and reasons, in a wide forum interactions. variety of ways. 3. Resource use can include consultation of documents HDOUQLQJIRXVU\HYDWQHHVUS  DOWHUUEHHW9 DQD\OW - (manuals, web pages, slides), views of videos, et LERPPVDOOUÜIDPRQJULOQFWQVSLJRSDLDGEDKDRVGUFLV HO PG 144 HANDBOOK OF LEARNING ANALYTICS Figure 12.1.RUDJHUZIYYLHVEDGJHRHQSHULRGLYWÚHUZ$VWXGHQHUVRQDO%DGJH2YYLHGØ3YL%DGJHERDUD 1RS 7 (bottom) Navi Surface: students actively using the tabletop display application during a face-to-face session (Charleer et al., 2013). cetera. Techniques like software trackers and a node link diagramWRHQDEOHIXUWKHUH[USORWDQLRIR eye-tracking can provide detailed information O[SHFDQWVQHGWVXVWJKLQUHWKRJQ$RPVHJGDEHVHWK HUR DEWRXZKWDSDUWVIRUHVRXUFHVH[DFWO\DUHEHLQJ ZWKRVWLFKKHUHWXGHKQHDYVDUVSQHGHFLEDGDHāJFVV used and how. a means to compare and discuss learning progress. 4. Time spent can be useful for teachers to identify Figure 12.2 shows a dashboard that uses grades to students at risk and for students to compare their predict a student’s chances of failing a particular own efforts with those of their peers. FRXUVH2FKR DHUEHU9W&KLXOL]D 'XYDO EH - fore she starts. The dashboard is intended to support 5. Test and self-assessment results can provide an teachers in giving advice to students on their learning indication of learning progress. HVUHSGEDDRKGVUWHKSVHFLFāDO\OHUR0WMHFUUWDRHVL WVQ Figure 12.1 presents one of our more recent dashboards SDIDUWLKLWVVFWWLXQIHGRXODU  OLOHLNRKWRGHK DOLQJ 7.KH'OHU &N2XYKG DDU[O U/OXHLRHU]LROVD course in which she is interested. The dashboard uses dashboard tracks social data from blogs and Twitter. colour cues to indicate whether the risk of failure, G6FKXWDDFWDUHRJL]HDGV artefacts produced, is then based on past performance, is low (green), medium YLVXDUROIL]HG students. The goal is to support aware- WHPXRFRWHKQR'SHGQHLQJHU G KLJKUR OHZ\RO WHK ness about learning progress and to enable discussion teacher can advise the student to take the course or to in class. To support such awareness and discussion, HXUTHUSWDDāNUVWLQJDVKFXVDWUHOGLHVQWFVXDYLVV WLHLV social interactions of students are abstracted in the course. The dashboard also supports several interaction form of learning badges for students to earn. Students techniques that enable the teacher to indicate which H)DLJUHQGHKYDWHK\HGEDJVZKKFL[HHUROSWQHKFDQ - data should be taken into account to generate this XUH WRSWKURXJKWKHYLVXDOL]WDLRQRI icons and prediction, including sliders at the bottom that enable colour cues*UD\EDGHJVKHDYQRWH\WEHHQHDUQHG the teacher to specify the range of data in terms of 7KHEWRWPRSDUWIR)LJXUHVZKRVDYLVXDOL]WDQLR H\DUURH[V)DSOHPLIDVWWGXGHQLGSRURO\LQ%LRORJ\ developed for collaborative use on a tabletop that uses LQ*UDGHHURWNGXEZKDUGHUDQGGLHGOZOLQ*UDGH Figure 12.2. WDOFKRDH2RXUVH FLāFFIIDLOLQJDVSHHOLKRRGRWVWKHOLNHVHQHSUWUGWKDDGDVKERDUY0X PG 145 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS WKH*UPDGHDUNFDQEHGLVUHJDUGHG SFPRDUHGWWRRKHUVWWXGHQ"V • How much do I contribute to the discussion forum, HOW TO GET STARTED SFPRDUHGWWRRKHUVWWXGHQ"V In both cases, we deliberately only list questions that To leverage the advanced perceptual abilities of humans VWDUWZWLKÜÝZÝKÝÜWDDÜZKRZFKXKHQQGPÜZKRIR - HLJVGDWDSUWHQVUHGYLRVFDGQHU[HROSWPHKSOHKWR UHQ Ý7WHQKHVHVSHFLāFGLUHFWTXHVWQLRVFDQEHGLUHFWO\ must create a visual representation or encoding of the PDSSHGLQDGDWDVHW4XHVWLRQVOLHNÜ Why did this data (Card et al., 1999). Several steps, outlined below, student have to enroll twice in this course?DUHQVZWHKÝ can be distinguished in this design process. H[WUQSORWUWPRDHDUKR,LEQGWXDHQLLFPUVHW\\DDURV he did not spend enough time on the course material, Understand Your Goals did not interact with fellow students on the discussion PGRDLSUREOQHPWKHNHWJZQRWWRLLVQJVWāHSU7VWKH forum, started to study the course material too late, the data set, the intended end-users of the tool, the XRZODQUHVZWGRL$ITHXVWWRQQUHāFRLKQRXWOVRDGQ G typical tasks they should be able to perform, and so EHÜ Are students more eager to work on assignment on. The following questions need to be answered at 1 or assignment 2?ÝHQ(YLIFKXPGWDDLVFWSDXUHGWL this stage: GLLVIāFDWXTXHQRVVWOHUWZROLYYQLRQVXLKPQJPRDQ - 1. Why:WLRKWJDYVWIKHRDKHOLVXDO:L"]WDKQLRWD tivations based on a plurality of (un)known variables. TXHVWQLRVDEWWRXKHGWDDVKRXWDOGLQVHUZ" Especially in the early phase of design, it is therefore GLUHVSQFRHRWFIFLXāFVWIRDGHYRDDVWLLHUHQQGVDEOH 2. For whom"HGGQWHLQYXLVDQOWRLL]DWHKLVPRKZUR) questions. Are the people involved specialists in the domain, ULRQYLVXDOL"]WDQLR Acquire and (Pre-)Process Your Data Building a visual dashboard typically entails a data-gath- 3. What'R\GO"SDLVYXLVDOQLWWR]HLDKZLOWGODD:WKD HULQJDQGSUHSURFHVVLQJVWHS9LVXDOL]WDLQRH[SHUWV WKHVHGWDDH[KLELWDVSHFLāFLWQHUQDOVWUXFWXUH suggest that this step takes 80% of the time and effort OLHWNLPHUDQDKHRWLHUURNZD"\UFK (ODPT'YQGF V0Q RWHQHSOWRDLVHUVKHWUYOVXOV 4. HowRDJ"OYWLVXHZKSWSUVXRDHKLOWLOZ]WORQD+RL identify the following intermediary steps: How will people be able to interact with the vi- 1. Acquiring raw data: It is important to have a clear GHY"WRXLLWVFWXSXHQHQWGHDG:KHLOVL"K]WWDDQLR idea of where the data will come from (e.g., the log VWGDDHWKUWVHGGDLXJQQQGDQ[HDPLQLJQIFHDUXO\O\% WH āOHVIRWKH/06DVVHVWVPHQUHVXWOVWR KHUDQG a variety of questions about the data can be formed. when the data will be updated (continuously, not at Having these questions in mind can be useful when YDDLODEHGWDDW:HKLWLUHQOVSOHFL YDFāOVWDDOO HOE URG)DEKVRDWGUHKURIāGWWOUHDFTDXDLDLUGQQJLQJ [H - WKURXJKDQ$SSOLFWDLRQ3URJUDPPLQJ,WQHUIDFH ample, consider a data set that contains the following $ ,D3 QH[SURWāOHUVRWPHRRKHUV"RXUFH learner traces: 2. Analyzing raw dataHOFDHQGEHWRHQHGP\DW'DD • access to learning resources if some values are missing or er roneous, or • WLQSPHRDHLJQGLJWLDOWH[WERRNV pre-processed to compute aggregate values (mean, PLQLGPXWDPDDP,QQFD[HDW HULOHWD\XPVLPV • contributions to discussion fora distribution can also be an issue: are there apparent • time spent on assignments WRXOLHUVXFOVWHUVHWF"HWHUD )URPWKHVHWUDFHVZHFDQGHāQHVHYHUDOUHOHYDQW 3. 3UHSDULQJDQGāOWHULQJGDWD : Using the initial questions as a starting point in the design process. A questions from step 1, choose the relevant data teacher might ask questions like these: IPWURKHSRIDRORQDO\]HGUZGDWDD • When did students start looking at the course Mapping Design PWDHULD"O Important in the visual mapping design is to choose • What is the average time that a student spends a representation that best answers the questions you UHDGLQJWKHWH[WERRN" want users to be able to answer, i.e., that serve your YLVXDOL]WDLRQJRDORUIWKHLWQHQGHGWDUJHWDXGLHQFH • "WQJHLVVQPDVKLQRNURZUHWH3GGLXUVRK\PDQZR+ 7KHUHH[LVWVDXPWOWLXGHIRDWOHUQWDHLYV2QHZ\DWR • +RZRIWHQGLG3HWHUDVNDTXHVWLRQRQWKHGLV - start is to look at the measurement or scale of each FXVURVQILR"XP data characteristic. Nominal or qualitative scales differentiate objects based on discrete input domains, A student will probably ask similar questions: TXWRDFKHUOWDUHWDFLROURJVDWLVHDVFKHXVVLLYāFWDQLRV • How much time do I spend on an assignment, WRZKLFKWKH\EQHORJ4XDWQWLWDHLYVFDOHVKHYDFQR - PG 146 HANDBOOK OF LEARNING ANALYTICS WLXRXQVLWSXQGPRDLQVHJ @> 2UGLQDOVFD OHV between the numbers quite originally by relating have discrete input domains where the order of the them to age, where a person of 37 can easily lift WQWHHEHZHFQHGULIHI[HFWDWHKWXEWPUDWHVWVQHPHOH HK weights, while a person of 73 might already need YDXHOVGRHVWQR'HSHQGLQJQRWKHVFDOHIRWKHGWDD a walking stick. characteristic, one can choose how to encode this • )LJXUHGDGGXVPVFOHVL]H GDWDYLVXDOO\)LJXUHGHSLFWV0DFNLQOD\VÚ • FLFULGH]VXTHDO\ODQIRGKLDJVQHXVVHJ)LHXU HO ranking of visual properties to encode quantitative, with 75 versus 37 stripes. ordered, and categorical scales. For instance, the spatial position of an element is useful for encoding • Figure 12.4f: uses a position in a Cartesian coor- quantitative, ordered, and categorical differences. dinate system. This is why scatterplots have been used so often to • KVQRWLDOHUHORKZRWDUSWDXVDYHL]OVLJJL)HXU SL convey a variety of information. Length, on the other between the numbers. hand, can encode quantitative differences, but is of • Figure 12.4h: assumes a time-based relationship less value for encoding ordered and categorical dif- between both numbers, which leads to a negative ferences. Shape is at the bottom of the ranking for trend line. YLVXDOL]LQTXJDWQWLWDDHLUGHRYUHQGGGLIHUHQFHIVWXE is more often used to depict categorical data. • Figure 12.4i: uses point clouds. āG/HOLRZW\SURWRW\SHVVXFKDVSDSHUVHNWFKHVDUH • )LJXUHMYLVXDOL]HVDQXQEDODQFHGVFDOHWR often helpful during the design-mapping step. Figure represent a difference in weight. GHSLFWVDQHH[UFLVHJHLQYWRWKHSDUWLFSLDWQV IR • āZJWLWXKHKUHIRVL]H)WLUJFRHKOUHWDHXNVUH 9/W'2LDVXHZDDDXRUOQU$QLQÜ%<UQDJLOW\QHJKWLFV Ý WKHVL]IWHRXKHQPEHU W/WXUHRWDURKHJLDUDQWD$LQO]LHQQGJDO\6XWPLFPHUV HHVDUUGHGXOFLQD3WUDVSQLFWL /$, 6WWHWQV,LX UVHKF RUSW\LOHGāKJLKJQLGRFQHDOXVYLDJQLWFHOHVU$IHW WV\HSRW with good knowledge in learning analytics, but limited XEXYFVLLEHDQWOJLQVXDOLW ]RWURDDRO7EOHQLVORXDLHN WRWGHNDVHUHZ\7HKQYXRWDVLLOL]DWXREDHJGHOZNRQ HDN UOELYXLVDQ[OWHRLWL]LVDLQJUR OHF([IRRV0FULWQHYH DUHVL PLWXQHVWRHVWNFKDOOSRVVLEOHZD\VWRYLVXDOL]H D OLH3NURFHVVL QU'JRMV VLGSOHPWDWIXVRQDHWRPEZ7HU^HHUF[VKHL`VH Clearly some alternatives work better than others, illustrated to participants that from the moment they GHSHQGLQWQRJWFQKHRH[WXDOHLLZJHDW]KJWDQG QLR GLIEUWQRVDWWāFLLRVXDQHVWWVWUOYNWFKLURLLWPQVXJDO age) and the ability to be interpreted by users (e.g., the HQFRGLQJVIRGWDD7KLVLVUHĂHFWHGLQWXKHQPEHUIR mental model of a balanced scale). There is, therefore, sketches that two teams of two persons each were able QREHVWZD\WRYLVXDOL]HDGDWDVHWEXWVRPHWHFK - to generate in 15 minutes (see Figure 12.4a and 12.4b). niques have been proven to work better than others, By sketching, more ideas and questions about the data UHR[IDSOHP set are often raised, which in turn leads to new ideas • 3LHFKDUWVDUHXVXDO O\DEDGLGHHZD ) UYRILVXDOL]WUHDR[QLR)DSOHP • Bar charts can be quite powerful. • Figure 12.4c: participants represented the difference • &RUGRLQWDHGJUKSDVHQDEOHULFKH[USORWDQLR Figure 12.3.GHGHUHRUWLYWDWLWHULVWLFVRQTXDQDFWDFKDURSHURUGDIYLVXDOSUWLHVIDQNLQJR UV \Ú0DFNLQOD and categorical scales. PG 147 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS • J'UKSDLFIRVHWD\YFWQQRRGRHQDGG\QWLQLRDO • LVPLODUWHVLLāGQWRWDUSWHQVDGQHDYVXODSUP&RLQJ information and force the reader to deal with and differences. UHGXQGDQWDQGH[WUDQHRXVFXHV /HY\=DFNV • Sorting items based on a variety of data values 7HUY 6VNFKLD\QR or metrics. • Scatterplots and parallel coordinates are good • Filtering values that satisfy a set of conditions. representations for depicting correlations. In • WXRWVGDQHVXDYOFLāFHSVHPDNRWWGDDWLJJKQL+JOKL addition, Harrison, Yang, Franconeri, & Chang visually without making all other data disappear, (2014) found that among the stacked chart variants, DVLVWKHFDVHZWLKāWOHULQJGWDD WVWKHHEDDFNGVLUJQLāFDWQWRXO\SHUURIPHEWGRK WKHVWDFNHGDUHDDQGVWDFNHGOLQH(OOLRW • Clustering or grouping similar items together; for has presented a nice overview of these studies. H[DPSOHE\DJJUHJWDLQJTXDWQLWWDLHYGWDDHJ average, count, et cetera) to view it in a higher Documentation or lower level of detail. $VZLWKDQ\GHVLJQH[HUFLVHLWLVLPSRUWDQWWREH H[SOLFWDLEWRX • $QWQRWDLQJāQGLQJVDQGWKRXJWKV 1. Rationale: W hy were certain decisions made, • %RRNPWDVSUHKHUHFYUNGRFQRDLULRLāLFHQZQJJ ZKWZDDVWKHLWWQ"HQ data to enable effective navigation. 2. Alternatives: Which alternatives were considered Heer and Shneiderman (2012) is essential reading on DHUHWQGZ\ZKKHWZ\QRWLK"KHOG interactive dynamics for visual analytics. The authors QFRG\WLWQUHQKFDWDWDHPLYFLVIRW\DPRQR[DHUWSVQH - 3. Evolution: How has the design evolved from early tribute to successful visual analytic tools. For each HVWNFKHVWRDIXEOZROOQLSPOHWPHQ"WDLQR:KWD WDVNFWDUHRYJDU\LRXVH[LVWLQJYLVXDOL]WDQLRV\VWHPV URIZKWDDGQHUDQVVWRSHFXQRFDOURIRPGLHāGZDV are described with useful interaction techniques that implementation or other reasons (logistics, lack support the task at hand, such as brushing and linking, IWRLPHWRKHUUH"D VQRV KLVWRJUDPVOLGHUV]RRPDEOHPDSVG\QDPLFTXHU\ Add Interaction Techniques āWOHUZLGHWJVVPDOXOPWOSOHLGLVSO\DVURWUHOOLVWSORV 9LVXDODQDO\VLVW\SLFDOO\SURJUHVVHVLQDQLWHUDWLY H multiple coordinated views, visual analysis histories, UHāQHPHQDHG[QURWOFSDUHQWRLYDQRLHLZIRURSFHVV W and so on. H+HU 6KQHLGHUPDQ %URHIHDQDO\]LQJZKLFK Evaluate Continuously LWQHUDFWQLRWHFKQLTXHVDUHXVIHXUORDIVSHFLāFYLVX - FUQRFIRQWRUOLDEHDRWHKRHFVUVSLJHVGQWHK'XULQJ WHH DOL]DWLRQDSSOLFDWLRQLWLVXVHIXOWRXQGHUVWDQGW KH personas and scenarios can be very rewarding as it typical analytical tasks performed by teachers who helps to focus the design, development, and evaluation want to understand how their students are doing in HUHYDVL\\VYUWHOH,YWDLLWKHVQXVZIDRKWDOQLR]WDQLR FODVV6HHUYDOWDVNWDQR[RPLHVKHDYEHHQGHVFULEHG ODGQF\DGQÝH\HÜKFXPWRRZWKL\DZDFDUUHGLWHJWR HRV in literature for this purpose. Common tasks include: YLXVDWHOKLW]D\ZKDGQPRZKURIZKWDWHKIRWUNFD QRL Figure 12.4. Sketches of a small data set of two numbers {75, 37}. PG 148 HANDBOOK OF LEARNING ANALYTICS LVEHLQJGHVLJQHG*HQHUDOO\DXFHWVQHUUHGGHVLJQ CONCLUSION WQHPSHORHGYWLZHUUSRWDFHHWLLHYGKVUSSRDKFD &'8 that keeps the target users in the loop in continuous JRORHWRLGHKPDGQWSVHFQRFYXLVDQOWLRL]DUQRWRPLQ,DI V FFOH\VIRGHVLLJWSOHPHPQ×W×HQDYDQLRXOWDQLR,QWKLV are key enablers for way, the development can focus on the most relevant • Learners to gain insight into their learning actions issues for teachers or learners at all times. and the effects these have. 7KHHYDXOWDLQRIRLQURIPWDLQRYLVXDOL]WDLQRV\VWHPV • Teachers to stay aware of the subtle interactions is essential. A plethora of techniques can be used, in their courses. LQFOXGLQJFWRQUROOHGH[SHULPHWQVWKWDHYDOXWDHGLI - IHUHQWYLVXDOL]DWLRQDQGLQWHUDFWLRQWHFKQLTXHVRU • Researchers to discover patterns in large data āHVOGWXGLHWVKDWDVVHVWVLKHSPDFYDIRWLVXDOL]WDQLR sets of user traces and to communicate these QROHDUQLQJ O3DLVDWQ 7KHOWDWHUWDHNSODFHLQ data to their peers. natural environments (classrooms) but are often time YXQLVLXWDHKKODLVKF]WSWWDUHDQLRKLLVQZVRK$QV THX FRQVXPLQJDQGGLIāFXOWWRUHSOLFWDHDQGHJQHUDOL]H potential to help shape the learning process and 1DJHOHWDO HUE9HUWHWDO  VXJJHVW WKH HQFRXUDHJUHĂHFWLRQRQLWVSURJUHVVDQGLPSDFWE\ following evaluation techniques: creating learning analytics dashboards that give a 1. Effectiveness, which can refer to engagement, concise overview of relevant metrics in an actionable higher grades or post-test results, higher reten- Z\DDQGWKWVSSXDURWWKHH[USORWDISWQRLRDWHUQV tion rates, improved self-assessment, and overall 'HVLJQLQJDQGFUHWDLQJDQHIHIFWLHYLQRUIPWDLRQYL - course satisfaction. VXDOL]WDVQLR\OHVURWIDHPUQLQDJQDO\WLFLVDVDQUW DV 2. HDOUHQURUHKFWHDDIRWHLPIRHXVHWKLQFQI\(HLFā U HODUQLQQ[HRSUHPRDGWLLQVHWERKHHQGVHLGVJUHQWHK J theories and paradigms as well as techniques rang- 3. Usability and usefulness evaluations often focus ing from visual design to algorithm design (Nagel, on teachers being able to identify learners at risk 6SHQFH ,QWKLVFKDSWHHUZKHDYEULHĂ \ or asking learners how well they think they are YLVXDDOGHLYVLLDQJV]WUWWHDSLQVLRWQKHQLXRURVFXHGG performing in a course. process, from raw data analysis to effective dashboards Typical evaluation instruments include questionnaires evaluated by target users. UHWURUWWRDNVLHPHUHZ[KHSUHWWORQLUQRFHHVOPGUR V PWOHDGHWLRDPHUFHWDHHWUHQYDUDHXOW DH'GLOOHQ - bourg et al., 2011). REFERENCES WDVHVWXGHQH\DUQLQJDQDOWLFVWRLQFUVLQJOHGXH8W3XULJQDOVD  &RXUVH6$UQROG.( 3LVWLOOL0' nd success. Proceedings of the 2 International Conference on Learning Analytics and Knowledge (LAK ’12), 29 &0N$RUZ<H 1×%&&DQDGD SSHUYRXDQF\90D$SULO× WLRQDFWHUWLQDOSDUIWHPSRUWLFLSDQZRHYLH\VHGÚ3$ELUZVRQ6  61$3D%DNKDULD$ ' Proceedings of st the 1 International Conference on Learning Analytics and Knowledge /FK0DU$.Ú )HEUXDU\× &0N$RUZ<H 1%DQII$%&DQDGD SS× QWLRQ,DGXFGHHEEDVHWWXWRULQJV\VWHPVWRZWHOOLJHQRPLQGLD)U\SHUPHHK $GDSWLY 3YVN\UXVLOR% th /HKQ (GV DQDVVRQ.9&)UXWKLHU*D* Proceedings of the 5 International Conference on Intelligent Tutoring SystemsGRLSULQJHU 6×DO4&&DQDGD SSHWURQ0-XQH × ,76 6KQHLGHUPDQ% (GV  -\G6.0DFNLQOD&DU Readings in information visualization: Using vision to thinkXIPDQQ3XEOLVKHUVJDQ.DRUOLQJWRQ0$0XU% RXJWLRQWKUFHĂHHQHVVDQGUKDUZYLQJDRPSU ,DO( Y 'XROD6/XLV-GULR]2N[-6.OHUHUOH&KDU DQ/3DPPHU3H9RRURJVWLH$0FLN%.UYDQ0.UIEDGJHV,WLRQVRHYLVXDOL]DWLYDFWHUHLQWLYDROODERUF - rd 8OOPDQQ (GV 'W 7GHLQKDU5QHVH03ULOOD: Proceedings of the 3 Workshop on Awareness and Re- QKDQFHG/HDUQLQJHFKQRORJ\(ĂHFWLRQLQ7 DSKRV&\SUXV SSSS×  6HSWHPEHU37(/Ú$5 W6 .DSODQ)  &XHQGHG4'ROHQK6%RQQDUHUPDQQ3YL+-$OD*\H=XIHUIJ3'LOOHQERXU th Classroom orchestration: The third circle of usability. Proceedings of the 9 International Conference PG 149 CHAPTER 12 LEARNING ANALYTICS DASHBOARDS on Computer-Supported Collaborative LearningRO,SSYRQJ&KLQD RQJ.\+&6&/ ×-XO HVDUQLQJ6FLHQFIWKH/HW\RWLRQDO6RFLHWHUQDQ× , GLPH W9LVKWSVW2SHQGDWHHVHQHSWLRQLQPLQXWHV3UF VWXGLHVDERXWKXPDQSHU(OOLRW.  - PLQXWHVIHDUV]FKLHSWLRQLQFVWXGLHVDERXWKXPDQSHUHQQHOOLRW#NWRPXPF H,4WLYFROOHWDQGERRVWLQJRXUFFWHOOHWLQJWKHKXPDQLQXJPHQGDDUZR(QJHO  7EDUW' Communications of the ACM, 38 × Few, S. (2009). Now you see it: Simple visualization techniques for quantitative analysis. Berkeley, CA: Analytics HVV3U FHĂHUHQHVVDQGVHOIDUZRUDWHUIW\PHWLYLWDFGR$  7KHVWXGHQDUDO( 3HUEHUYW.'XDHUWV69Y*R - tion. CHI Conference on Human Factors in Computing Systems: Extended Abstracts\0D&+,($Ú × &0N$RUZ<H$XVWLQ7 1;86$ SS× VHEHUÚWLRQXVLQJ:HODRUUIFWLRQVR 5DQNLQJYLVXDOL]DRQHUL6 &KDQJ5 DQFDQJ))UDUULVRQ/<+ law. IEEE Transactions on Visualization and Computer Graphics, 20  × \VLVRUYLVXDODQDO\QDPLFVIHGWLYDFWHUQ , 6KQHLGHUPDQ% )HEUXDU\-HUH+ Queue – Microprocessors, 10(2), 30. FHVLQSHUVSHHQFHUDSKLFV"3XWHIWRXVJUWXLWLQJSUD $SULO *U% 6FKLDQR'HUVN\7Y(=DFNV-Y\/H - tive. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems$SULO ×&+,Ú &0N$RUZ<H 1×%&&DQDGD SSHUYRXDQF9 WLRQRUPDWLRQDOLQIHODIUWLRQVRWDHVHQDOSUDSKLFIJUWLQJWKHGHVLJQR $XWRPD -\0DFNLQOD Transactions on Graphics, 5  × WLRQ$PRGHODQGLPSOHWLRQYLVXDOL]DRUPDVLQLQIGVXWLOL]LQJ*38DUZR 7 YLVW1'RQQHO% (OPTF0 - mentation of image-space operations. IEEE Transactions on Visualization and Computer Graphics, 15 DQVSRURXFKLQJWUW 7WFNO. 5DWL& .ORHN[-H$.OHURHUDQGH0DO(9YWDQ0'X0DLDJHO71 th WRSVHWDEOHWLYDFWHUWRQLQDQVLWDQSXEOLFWURSROLWU\RQYLVXDOL]LQJPHDVHVWXG$F Proceedings of the 12 International Working Conference on Advanced Visual Interfaces\ SSWDO\&RPR,0D× 9,$ &0N$RUZ<H 1× Nagel, T. (2015). Unfolding data: Software and design approaches to support casual exploration of tempo-spatial data on interactive tabletops. Leuven, Belgium: KU Leuven, Faculty of Engineering Science. NLQRU\DUQLQJDQDOWLFV:WLRQLQOHWHU\YLVXDOL]D 8QFWDLQDO( HUEHUYW.&KLOXL]D. 'XFKRD;92 progress for IEEE Transactions on Learning Technologies. nd WLRQDOXDYWLRQHWLRQYLVXDOL]DRUPDILQI 7KHFKDOOHQJHRW& ODLVDQ3 Proceedings of the 2 International Working Conference on Advanced Visual Interfaces \ SS×WDO\*DOOLSROL,0D ×9,Ú$ New York: ACM. 00LKDHVFX&/XQD-RV3WURQ$0LHUFHU00\FKHQL]NLHR&3RPHU5*DULR-&%RWLFWRV26DQ th DLV0 (GV   D6 'HVPDUWXUHQW]$9YLRHUVKN+HQR3RU0 Proceedings of the 8 International Conference on Educational Data MiningDWLRQDO(GXFWHUQDQSDLQ,-XQH0DGULG60 × (' - W\WD0LQLQJ6RFLHDWLRQDO' Shneiderman, B., & Bederson, B. B. (2003). HĂHFWLRQVUDQGHDGLQJVRUPDYLVXDOL]DWLRQ5WLRQLQIRIDIWFU7KH . Burl- XIPDQQ3XEOLVKHUVJDQ.DRULQJWRQ0$0 st Spence, R. (2001). Information visualization: Design for interaction, 1 ed. Salt Lake City, UT: Addison-Wesley. nd Tufte, E. R. (2001). The visual display of quantitative information, 2 &HVV//DSKLFV3UH&7*UG&KHVKLUH GV$QDUQLQJGDVKERDU /H N[- .OHUD*DUU$VVFKH)3WRV-DO(6DQDHUYHUEHUWV6'XYW.*R9 overview and future research opportunities. Personal and Ubiquitous Computing, 18 × nd Ware, C. (2004). Information visualization: Perception for design, 2 ed. Burlington, MA: Morgan Kaufmann 3XEOLVKHUV PG 150 HANDBOOK OF LEARNING ANALYTICS

Journal

Handbook of Learning AnalyticsUnpaywall

Published: May 1, 2017

There are no references for this article.