class: inverse center middle title-slide ### ONE ARABESQUE IN THE SMALL WORLD OF OD WEBMAPS ##### <span style='color:#ef7d00;'>Nicolas Roelandt</span>, Françoise Bahoken, Grégoire Le Campion, Laurent Jégou, Marion Maisonobe, Boris Mericskay, Etienne Côme .bottom[ ##### <span style='color:#ef7d00;'>_FOSS4G 2021 - remote_</span> <img src="data:image/png;base64,#media/logos2.svg" width="100%" /> ] ###### [gflowiz.github.io/presentations/FOSS4G2021.html](https://gflowiz.github.io/presentations/FOSS4G2021.html) ###### Press <span style='color:#ef7d00;'>P</span> to access notes --- # Introduction .pull-left[ - Part of the Gflowiz project [geoflowiz.hypotheses.org](https://geoflowiz.hypotheses.org) ] .pull-right[ <img src="data:image/png;base64,#media/logo_gflowiz.jpg" width="40%" /> ] -- .pull-left[ - Focus on **web applications** ] -- .pull-left[ - Thematic *geovisualization* and cartographic purposes ] -- .pull-left[ - **Flows** and spatial networks ] -- .pull-left[ - Review of existing tools (2016 -) ] -- .pull-left[ - Survey of 200 users ] -- .pull-left[ - Development of Arabesque (2018 -) ] ??? The paper, as the Gflowiz project, focuses on a specific part of geoweb, it relates to flows and spatial networks for thematic geovisualization and cartographic purposes. Arabesque, after an online survey of people interested in geovisualizing this type of data and a review of existing web pages and applications. Particularities of flow mapping : pay close attention to the preparation and processing of the OD data before representation, otherwise graphical (e.g. spaghetti effect) and visual complexities will arise. arabesque positioning: combining OD data visualization and cartography (not also mapping). 3 key points: dealing with numerical data, geographical data, display/cartography. --- class: inverse center middle # Why compare ? --- layout: true ## Why compare ? --- - Determine the strengths and weaknesses of arabesque, compared to other applications -- - What is Arabesque's position regarding similar applications ? -- - Effort on rendering and data analysis still innovative ? -- - Can we help users to make an informed choice ? ??? Arabesque is now an usable application, useable by everyone. Is Arabesque really innovative ? This paper shows what Arabesque's positioning is: we insist on the role of upstream data processing (stats. and geo) and not only on representation aspects, on rendering... because the team work with statistical data. An effort that has been made on the rendering, the arabesque representation part is very developed: all the dimensions of the flow signs are customizable (their geometries as well as their appearance via the possibility to combine several visual variables...) But is Arabesque really innovative, does it brings something to the community that are not in similar tools ? It is not perfect, some features we think important are missing, there is some inconvenient bugs. We are working on a version 2. So we made this comparison and wrote this paper to help user make an informed choice --- layout: false class: inverse center middle # Comparison criteria --- layout: true # Comparison criteria --- Focus on applications similar to Arabesque : * Web applications (online only) * Free and Open Source Software * Origin-Destination data visualisation capabilities * Ability to use user's own data -- 4 apps: - Arabesque - Flowmap.blue - Kepler.gl (Flourish) - Magrit ??? We decided to focus on applications that are similar to Arabesque: - Web applications that runs in a web browser (so no desktop application like QGIS and no programmation languages like Python or Javascript) - Free and Open Source Software for 2 reasons: Arabesque is FOSS and for reproductibility concerns of the article, we choose to only compare it to software that are accessible - The softwares has to be able to visualize Origin-Destination data, a capability often lacking in webmapping tools - An important criteria was to be able to use our own data so we can compare the same dataset - We used the same subset of the MOBSCO dataset in each application - First we decided to compare Arabesque to Flowmap.blue, Kepler.gl, Magrit and Flourish. Flourish is based on the same technologies as Kepler.gl, and does not bring much more than a pleasant interface. We have therefore chosen to discard it. --- layout: false ## Arabesque .left-column[ * Launched in 2019, part of the [Gflowiz project](https://geoflowiz.hypotheses.org/) * Exploration and geovisualisation of origin-destination flows] .right-column[ <img src="data:image/png;base64,#media/image3.png" width="1000px" /> ] ??? Arabesque was developped in 2019 inside the Gflowiz project with the goal to provide a tool dedicated to exploration and geovisualisation of origin-destination flows and spatial networks. Few tools are able to map origin-destination flows and often lacks functionnalities like - advanced cartographic and styling tools (you can't change the shape of the link for example) - data analysis oftel has to be before hand and can't be done within the app --- ## [Flowmap.blue](https://flowmap.blue/) .left-column[ * Created in 2019 by [Ilya Boyandin](https://ilya.boyandin.me/) * Explore and create flow maps * [flowmap.blue](https://flowmap.blue/) ] .right-column[ <img src="data:image/png;base64,#media/image5.png" width="600px" /> ] --- ## [Kepler.gl](https://kepler.gl/) .left-column[ * Launched in 2018 * Geovisualisation tool designed for complex datasets * [kepler.gl](https://kepler.gl/) ] .right-column[ <img src="data:image/png;base64,#media/image4.jpg" width="1000px" /> ] --- ## [Magrit](http://magrit.cnrs.fr/) .left-column[ * 2017 by the RIATE team * Thematic cartography software * [magrit.cnrs.fr/](http://magrit.cnrs.fr/) ] .right-column[ <img src="data:image/png;base64,#media/image6.jpg" width="1000px" /> ] --- layout: false class: inverse center middle # Data model --- layout: true # Data model --- * Data format : mostly CSV files -- * Data storage : wide/large matrix format more frequent -- * Nodes coordinates in WGS84 (lat/lon) -- * No cleaning tools provided ??? Data format: - 2 files nodes / links is the most common (all but Kepler.gl). - kepler.gl works with a single file with Origin and destination coordinates in 4 columns Coordinates: Coordinates has to be in WGS84 for Arabesque, Kepler.gl and Flowmap.blue Magrit accept projected data So the data might need to be transformed before hand Cleaning: Your data needs to be prepared ahead using other tools. Arabesque provide information on rejected data. --- ## [Mobsco](https://www.insee.fr/fr/statistiques/4509360) dataset - French student mobilities in 2017 - Elementary school to grad school - 4,730,521 records - 27090 records aggregated at *département* level ??? The MOBSCO is a dataset produced by the national statistics bureau [INSEE](https://www.insee.fr/en/accueil). It compile more than 4 millions records of student mobilities from home to school, from elementary level to grad school. This is too much for some applications so we aggregated it to the French *département* admnistrative level --- layout: false class: inverse center middle # Usage --- layout: true # Usage --- .pull-left[ * Representation * Arabesque* * Flowmap.blue * Kepler.gl * Magrit* ###### * advanced graphics ] .pull-right[ <img src="data:image/png;base64,#media/app_mosaic.jpg" width="1200px" /> ] ??? All applications can be used to represent OD data. However Kepler.gl and Flowmap.blue offers limited design options (3D arcs). --- .pull-left[ * Exploration * Arabesque* * Kepler.gl * Flowmap.blue* ###### * multiscalar visualization ] .pull-right[ <img src="data:image/png;base64,#media/arabesque_240p.gif" width="1200px" /> ] ??? Magrit is more focused on the production of static maps and does not allow navigation and exploration kepler.gl and Flowmap.blue provide some exploration functionalities like zooming on a region, popups on hovering or brushing. Arabesque does not provide exploration functionnalities like po-ups windows or brushing, but provide tools for data analysis and exploration. Note that Arabesque and Flowmap.blue offer multiscalar ("scalable") visualization capabilities. There is a resizing/adaptation of the figures with respect to the level of observation: it's not a zoom/dezoom (knowing also that zooming is not changing the scale) --- .pull-left[ * Data Analysis / Exploration * Arabesque * Kepler.gl ] .pull-right[ <img src="data:image/png;base64,#media/arabesque_data_exploration.gif" width="1200px" /> ] ??? Magrit data analysis capabilities are limited to a histogramm of the dataset without filtering. Arabesque and Kepler.gl offer data analysis capabilities with custom filtering options. Arabesque also compute new indicators (inDegree, OutDegree, balance). --- layout: false class: inverse center middle # Public/Purpose --- layout: true # Public/Purpose --- * Data exploration and visualisation * general public * data journalist * web publishing * **Kepler.gl** / **Flowmap.blue** --- * Academia : research and teaching * thematic maps production * statistical analysis capabilities * **Arabesque** / **Magrit** ??? Which application to use for which purpose? Each tool meet specific needs according to the profiles (technical skills, mapping knowledge) and objectives (communication, exploration, mapping) of the different users (geographer, cartographer, GIS specialist, data scientist, data journalist, urban planner, etc.). We think that for data exploration and visualisation, by the general public or data journalist, kepler.gl or Flowmap.blue are best fitted. They do not require the user to have significant knowledge of cartography and are oriented towards web publishing, allowing quick publication and interactive reading. On the other hand, the representation possibilities are limited to a colored 3D arc. For the creation of static thematic maps or statistical exploration of data, we recommend using Magrit or Arabesque. The representation possibilities are more important, although limited to two dimensions. The result will be more suitable for teaching or publishing scientific articles. --- layout: false class: inverse center middle # Performance --- layout: true # Performance --- ## Technology dependant -- * Kepler.gl and Flowmap.blue * Deck.gl * large datasets * fast * limited graphic options -- * Arabesque and Magrit * D3 * smaller datasets (< 10 000 objects) * more styling options ??? The performance is also very good of Kepler.gl and Flowmap.blue, allowing to visualize large datasets, quickly. Arabesque and Magrit do not support large datasets under penalty of slowing down or blocking your computer. It may therefore be necessary to aggregate the data beforehand. --- layout: false class: inverse center middle # Interoperability and reproducibility --- layout: true # Interoperability and reproducibility --- ## data format * CSV and/or GeoJSON * 2 separates tables (except for Kepler.gl) * same files for Arabesque, Magrit & Flowmap.blue ??? Most applications tested used CSV files as input, some accept geojson. Generally they except 2 separates tables Note that for Flowmap.blue you will have to save the CSV file in a google sheet document or copy-paste the data in a browser --- ## Export * Online hosting (Kepler.gl, Flowmap.blue) * Image export (Arabesque, Magrit) * Data and project parameters (Arabesque, Kepler.gl, Magrit) * allows project reuse * allows the processing reproducibility * important in the context of Open Science ??? Kepler.gl and Flowmap.blue propose to the user to host online their production, which is a real asset to diffuse dynamic cartography online. Arabesque and Magrit are more oriented towards editing cartography and only offer image output formats. Arabesque, Kepler.gl and Magrit allow you to export the data and parameters of a project so that a user can share their analysis. Those capacities to export and load saved project is important for the user but it is also important for collaboration. It allows the reproductibilty of the treatments used to produce the map and therefor are important for Open Science --- layout: false class: inverse center middle # Conclusion --- layout: true # Conclusion --- - Business culture, user's skills and final use of the map influence the choice of the tool because it will influence the chosen approaches (exploration and/or analysis) - Data does not influence the choice of the tool - A whole range of needs and possibilities are covered, for the benefits of the community. --- layout: false class: center inverse # Thanks! Nicolas Roelandt - Univ. Gustave Eiffel [nicolas.roelandt@univ-eiffel.fr](mailto:nicolas.roelandt@univ-eiffel.fr) [@RoelandtN42](https://twitter.com/RoelandtN42) Access to Arabesque application : [arabesque.ifsttar.fr](http://arabesque.ifsttar.fr/) Article pre-print version : [gflowiz.github.io/article_foss4g2021](https://gflowiz.github.io/article_foss4g2021/articles/article_FOSS4G2021.html) .center[ <img src="data:image/png;base64,#media/logos2.svg" width="100%" /> ] .bottom-left[ ##### Slides created via the R packages [xaringan](https://github.com/yihui/xaringan) and [gadenbuie/xaringanthemer](https://github.com/gadenbuie/xaringanthemer) ]