Outcomes

The practical outcome of MOSAICO will consist in a proof-of-concept demonstrator that will allow a dynamic deployment of micro-services, at different layers by a multi-layer orchestrator, to offer a global service ensuring high levels of QoS, security and reliability for one of the targeted use-cases. More precisely, the outputs of the project will be:
 

1. First and foremost, the scientific advancement of knowledge that will be carried out by all partners who, through high-level publications, will enable the scientific community related to Network Virtualization, Network and Service Management, and Security to benefit from the scientific results of the project;
2. The production of micro-services that aim at delivering contents given the low-latency requirements and at securing and monitoring the network so as to avoid any delay;
3. Novel multi-layer orchestration algorithms for selecting the best placement, technology and service chaining so as to deploy the necessary micro-services according to the service description and requirements; d) Some extensions of an open-source orchestrator which include the multi-layer capacity and the novel algorithms, and the possible contributions to the related open-source community;
4. The implementation and evaluation of the whole solution for a specific use-case. Modules will be designed, implemented and evaluated in conditions as close as possible to a real deployment (e.g. use of P4- based hardware, public open-source orchestrator, Montimage monitoring platform).

Meeting reports

 

#Meeting	Date	Location	Link to document
#14	29/03/2023	Rennes	Meeting Report #14
Abstract: The meeting #14 took place at IMT Atlantique in Rennes, March 29 2023. An orchestration solution has been presented. New heuristic methods were exposed, showing different efficiency. This was followed by a presentation of the testbed regarding security tests and some virtualization limitations led us to use a hardware testbed instead. A demo about a homemade cloud gaming plateforme was then exposed, using SCReAM. Then an architecture for cloud gaming detection, relying on P4 and NFV, was exposed, showing that USAD (unsupervised leanring) works better than DT (supervised learning). Then an analysis on celullar AQM and transmission opportunities was presented. The meeting was conclued on a discussion about the final testbed, regarding the integration of the different aspects of the project.

 

#Meeting	Date	Location	Link to document
#13	06/12/2022	Paris	Meeting Report #13
Abstract: The meeting #13 took place at Orange in Paris from December 6 to December 7, 2022. A orchestration solution has been presented, more specifically existing heuristic methods and meta-heuristic methods to solve the placement problem. This was followed by a presentation of actual Common Vulnerabilities Exposures (CVE) related to time-delay attack. A demo about base station instrumentation in a OAI context was then exposed, it includes EnhancedBSR, a prediction model that helps sharing the allocation of emitting time-slot. Then, discussions about the testbed occured. Those discussions covered security and detection, OAI station integration and orchestration. About the cloud gaming, a comparison AI-assisted anomaly detection solutions was presented showing interesting results, also regarding the security point of view. Then tests about QoE in cloud gaming was performed and presented to conclude this meeting.

 

#Meeting	Date	Location	Link to document
#12	05/10/2022	Nancy	Meeting Report #12
Abstract: The meeting #12 took place at LORIA in Nancy from October 5 to October 6, 2022. A 5G-in-a-box testbed has been showed. This rely on open source hardware and software and will be used to perform wireless tests about cloud gaming and VR. This was followed by a presentation of a testbed integrating L4S in a P4 environment. A demo about monitoring abilities through INT demonstrated the relevance of this approach. Then a review of ML techniques that can detect session anomalies has been presented, followed by a review of CG-centric CCA and AQM. A discussion about Horizon Worlds then occured. The orchestration model and its evaluation has been also discussed. To conclude this meeting, a review of detection techniques targeting low latency treats has been presented.

 

#Meeting	Date	Location	Link to document
#11	15/06/2022	Lannion	Meeting Report #11
Abstract: The meeting #11 took place at Orange Lab in Lannion from June 15 to June 16, 2022. Three papers have been submitted for cloud gaming detection, low latency threat characterization and a model for microservices orchestration. Research directions for threats targeting low latency services using L4S have been presented, followed by a discussion about the integration of the security detector within a Inband-Network-Telemetry architecture. Then the consortium focused on an evaluation of SFC placement for best effort traffic and Low-Latency traffic. AI-assisted cloud gaming detection has been discussed, more specifically the quality and characteristics of the data to inject in the model. A presentation of P4 for hardware occurred. A preliminary study on quality degradation diagnosis for cloud gaming traffic has also been discussed. The consortium initiated the elaboration of the deliverable 1.2 of the project.

 

#Meeting	Date	Location	Link to document
#10	23/03/2022	Rennes	Meeting Report #10
Abstract: The meeting #10 took place at IMT Atlantique in Renne from March 23 to March 24, 2022. The meeting began on the topic of cloud gaming tests under cellular network conditions followed by L4S tests under cellular network conditions. Then a live classification of cloud gaming traffic using python and P4 has been presented. The consortium then focused on L4S monitoring with Inband Network Telemetry and P4, leading to a discussion about microservices implemented in P4. This led to discussed about microservices orchestration. The consortium closed the meeting on the topic of the network general architecture of the MOSAICO project.

 

#Meeting	Date	Location	Link to document
#9	02/12/2021	Paris	Meeting report #9
Abstract: The meeting #9 took place at Orange Lab in Paris from December 02 to December 03, 2021. A first version of the orchestration model has been presented. This model is able to dynamically reconfigure Service Function Chains, implying parralellism and mutualization. Then a presentation about monitoring and closed-loop has been made, implying In-band Network Telemetry. We discussed about the position of the high precision monitoring in the network and the possibility of using LatSeq in a 5G architecture conjointly used with an adaptation of L4S. We also discussed about 3 closed-loop use cases: latency control with L3-VPN, thoughput control, extended IGP metric. Then presentation about undesirable flow detection occured, followed by the cloud gaming traffic detection.

 
 

#Meeting	Date	Location	Link to document
#8	30/09/2021	Nancy	Meeting report #8
Abstract: The meeting #8 took place at Loria in Nancy from September 30 to October 01, 2021. The consortium launched the writing about the deliverable D234. A presentation about High-precision monitoring has been made by Montimage, involving In-band Network Telemetry and Big Packet Protocol. Montimage also presented an anomaly detection and root-cause analysis technic using P4. Then, UTT expose the results of the published paper in HipNet Workshop about L4S vulnerabilities. The UTT also presented the orchestration model proposed to handle microservices in a heterogeneous context. The Loria proposed a network behavior analysis to classify cloud gaming traffic among other services and a local demo has been made to expose a home-made local cloud gaming platform.

 
 

#Meeting	Date	Location	Link to document
#7	08/07/2021	Jitsi Meet	Meeting report #7
Abstract: The meeting #7 took place Online on July 08, 2021. The consortium discussed about L4S monitoring and associated metrics and indicators. Then we focused on the behavior of a cloud gaming traffic under perturbation and we conclude that nowadays proposed platforms do not react the same way depending on the type of perturbation. The split of the MMT into microservices has been presented and implemented with OpenNetVM, but performance issues has been encountered. A presentation about L4S under undesirable flows brought some evidences about L4S vulnerabilities. The consortium discussed about attack scenarios. To conclude, we discussed about the orchestration model and raised the question about whether or not the delay should be seen as a constraint or as a target.

 
 

#Meeting	Date	Location	Link to document
#6	26/05/2021	Jitsi Meet	Meeting report #6
Abstract: The meeting #6 took place Online on May 26, 2021. The consortium has discussed about L4S tests on a P4 implementation. This work is done in variable network conditions, made with a scheduler that emulates the emitting pattern of cellular networks, following Mahimahi traces format. Then, a presentation of L4S vulnerabilities has been made and several potential undesirable flows and attacks has been identified. A presentation of Google Congestion Control has been made and the consortium discussed about the evolution and integration of ECN and L4S in GCC/WebRTC. A first model of multi-layer orchestration has been presented. This leds us to the question of how to split classical service chaining into microservice chaining, and for what adventages (such as mutualization). The description language to choose is also a key-concern. Cloud gaming tests has been presented,  in order to extract some key feature to facilitate traffic characterization. A discussion on the monitoring occured and the idea of splitting the Montimage Monitoring Tool (MMT) into micro-services has been proposed as a use case for orchestration.

 

#Meeting	Date	Location	Link to document
#5	11/03/2021	Jitsi Meet	Meeting report #5
Abstract : The meeting #5 took place Online on March 11, 2021. In the meeting, the consortium discussed about the deliverable D1.1 and how to use it to submit a survey about low-latency services and orchestration. The state of the art in orcherstration is made and will help to make mathematical models in a NFV context with microservices environment and with end-to-end latency as a constraint. A discussion about QoS in relationship with L4S brought several questions about WebRTC for Cloud Gaming and competition between BBR (or GCC) flows and TCP Prague. A reflexion about common tools for MOSAICO leds us to decide to centralize each partner's code into GitLab handled by Loria and to use the Cybersec platform handled y the UTT with the help of Montimages' Monitoring Tool (MMT) which will soon be available in Open Source.

#Meeting	Date	Location	Link to document
#4	09/12/2020	Jitsi Meet	Meeting report #4
Abstract : The meeting #4 took place Online on December 09, 2020. In the meeting, the consortium identified relevant micro-services, use cases and trafic classes. Then, a survey on Cloud Gaming has been presented by Loria. And an explanation about dynamic service chaining within OpenNetVM has been discussed. The consortium annouces disseminations of MOSAICO project in RESSI. Tasks have been distributed among partners to constitute the deliverable D1.1.

 

#Meeting	Date	Location	Link to document
#3	30/09/2020	Troyes	Meeting report #3
Abstract : The meeting #3 took place at University of Technology of Troyes, in France on September 30, 2020. In the meeting, the consortium presented ONAP and OpenMANO for MOSAICO orchestration. Then, OpenNetVM, a framework for micro-service orchestration has been presented. A first propositon for the global MOSAICO architecture has been formulated. A reflexion about use cases, their classification and low-latency services has been collectively done. The consortium annouces disseminations of MOSAICO project in Cloud Days.

 

#Meeting	Date	Location	Link to document
#2	12/03/2020	Troyes	Meeting report #2
Abstract : The meeting #2 took place at University of Technology of Troyes, in France on March 12, 2020. In the meeting, the consortium presented a P4 demo and the ONF ecosystem (switch, P4, Stratum, Onos, etc...). Then a presentation on existing microservice solutions and orchestration algoritm occurred.

 

#Meeting	Date	Location	Link to document
Kick-off Meeting	2/12/2019	Chatillon	Kick-off Meeting report #1
Abstract : The Kick-off Meeting  took place at Orange Labs Chatillon, in France on March  02/12/2019. The aim of the Kick-off of the MOSAICO project is for the contributing members of the project to get to know each other and to be at the level on the objectives and tasks of the project. Orange presented the overall context and objectives of the project. Cutting and organizing tasks were then introduced. Communication tools (through the setting up of messaging for project members and the creation of a website) and the operation of the project were also discussed. As for work tools It was decided to draft the Latex deliverables. Loria will set up a ShareLatex project for the MOSAICO project. It is also planned to have a GitLab for document sharing. Loria will set up a GitLab for MOSAICO.

 

Deliverables and reports

#Delivrable	Title	Release date	Link to document
# D3.1	Optimisation of Micro-services Placement and Chaining for Low Latency Services	23/11/23	Delivrable D3.1
Abstract : This document addresses the major challenges of modern networks in terms of flexibility, scalability, and, above all, low latency in the era of network function virtualization and the emergence of low latency applications. It presents an optimization model for the placement and chaining of micro-services, particularly in the context of low latency Service Function Chains (LL SFCs). This model follows technological considerations of the MOSAICO project regarding packet forwarding strategy, adopting the L4S approach. The document also explores cohabitation strategies for LL and BE (Best Effort) SFCs, proposes a heuristic approach as a fast and realistic alternative solution, and presents specific use cases illustrating practical applications of these concepts. Finally, it analyzes the performance and efficiency of these models through various evaluation scenarios, offering valuable insights for future developments in this field.
#3 D234	Selection, benchmark and design of the individual components forming the core of the global MOSAICO low-latency architecture	05/02/23	Deliverable D234
Abstract : The present deliverable stands for the mid-time milestone of the MOSAICO project, started on December 1, 2019. As such it gathers the set of achievements produced over the last two years period. As initially planned, the project has followed a practical approach with the objective to first evaluate some operational situations which can further guide the design and implementation of subsequent components. As such, the project selected cloud gaming as the main use-case for guiding all subsequent works and L4S as the core framework for packet forwarding. In band telemetry, possibly implemented with P4-based network equipment, and closed-loop frameworks for monitoring and troubleshooting solutions are envisioned but still under investigation. In order to reflect at best the works achieved during this first period as well as those currently ongoing, the deliverable presents all the MOSAICO contributions according to: (1) their maturity in the project advancement; and, (2) their fulfilment of an operational aspect we need to address, whatever its belonging to a particular task.
#2 D1.2	Final Description of Micro-Services and Preliminary Architecture	16/02/23	Deliverable D1.2
Abstract : Following the review of low-latency (LL) services and candidate technologies achieved in deliverable D1.1, the present document aims at exposing the functional description of the micro-services selected by the project and shape the general architecture which will host them. As such, this deliverable D1.2 first exposes the general architecture we envision consisting of three topological levels, from the local one up to the national one. The latter is refined with an analysis of the different deployment levels that a node can implement within this global architecture. Indeed, two types of micro-services are considered in the project: (1) those directly focusing on data-plane processing and deployed on P4 hardware devices, and (2) those requiring more resources and deployed on control-plane through lightweight virtualization. Then, the deliverable exhaustively specifies the expected functionalities of the different micro-services we will implement and deploy. These micro-services are related to packet processing, monitoring and security of the architecture. Finally, some prospective use cases, which represent relevant industrial scenarios, involving network function disaggregation are exposed to identify further developments of the project.
#1 D1.1	Low-latency: applications, network solutions,attacks and optimisation techniques	01/06/21	Deliverable D1.1
Abstract: The MOSAICO project aims at defining networking solutions for the secure and efficient deliveryof low-latency applications, such as cloud gaming, cloud robotics, drone piloting, etc.. This first deliverable presents a survey and our analysis of all the key topics addressed by the project as well as a first selection of the candidate technologies. First, the different classes of low-latency applications are described, mainly focusing on their main characteristics and their latency considerations. The sources of latency for delivering a service from the server to the end-clients are presented, highlighting the network ones, which we will deeply investigate and improve subsequently. To that aim, the deliverable introduces the main current network solutions for addressing the latency issue: queue management and control congestion algorithms, and Network Virtualisation Function (NFV) orchestration for deploying network both monolithic and micro services. To ensure the right delivery of low-latency applications, the main threats which could have an impact on latency are presented. Finally, the two main technologies we advocate in the project for implementing and deploying microservices ensuring the secure and efficient delivery of low-latency services are presented: OpenNetVM, a microservice technology, and P4 (Programming Protocol-Independent Packet Processor), a network data plane programming solution.

 

Publications

• H. Magnouche, G. Doyen and C. Prodhon, "A Fair Sharing Approach for Micro-Services Function Chains Placement in Ultra-Low Latency Services," in IEEE Transactions on Network and Service Management, doi: 10.1109/TNSM.2023.3313647.
• J. R. Ky, B. Mathieu, A. Lahmadi and R. Boutaba, "ML Models for Detecting QoE Degradation in Low-Latency Applications: A Cloud-Gaming Case Study," in IEEE Transactions on Network and Service Management, DOI: 10.1109/TNSM.2023.3293806
• J. R. Ky, P. Graff, B. Mathieu, T. Cholez, « A Hybrid P4/NFV Architecture for Cloud GamingTraffic Detection with Unsupervised ML”, 28th IEEE Symposium on Computers and Communications (ISCC), 9 -12 July 2023, Tunis Tunisia
• H.N. Nguyen, B. Mathieu, M. Letourneau, G. Doyen, E. Montes de Oca, S. Tuffin, “A Comprehensive P4-based Monitoring Framework for L4S leveraging In-band Network Telemetry”, 36th IEEE/IFIP Network Operations and Management Symposium (NOMS), Miami, FL, USA, 8-12 May 2023 (not published yet)
• P. Graff, X. Marchal, T. Cholez, B. Mathieu and O. Festor, “Efficient Identification of Cloud Gaming Traffic at the Edge”, 36th IEEE/IFIP Network Operations and Management Symposium (NOMS), Miami, FL, USA, 8-12 May 2023 (not published yet)
• Marchal, X., Graff, P., Ky, J.R. et al. An Analysis of Cloud Gaming Platforms Behaviour Under Synthetic Network Constraints and Real Cellular Networks Conditions. J Netw Syst Manage 31, 39 (2023). 10.1007/s10922-023-09720-9
• M. Letourneau, G. Doyen, R. Cogranne, B. Mathieu, "A Comprehensive Characterization of Threats Targeting Low-Latency Services: The Case of L4S". J Netw Syst Manage 31, 19 (2023). DOI: 10.1007/s10922-022-09706-z
• J. Ky, B. Mathieu, A. Lahmadi, R. Boutaba, “Assessing Unsupervised Machine Learning solutions for Anomaly Detection in Cloud Gaming Sessions”, 4rd International Workshop on High-Precision, Predictable, and Low-Latency Networking (HiPNet), 18th International Conference on Network and Service Management (CNSM), Thessaloniki, Greece, 31 October - 4 November 2022
• H. Magnouche, G. Doyen, C. Prodhon, "Leveraging Micro-Services for Ultra-Low Latency: An Optimization Model for Service Function Chains Placement". In the 8th IEEE International Conference on Network Softwarization (IEEE NetSoft 2022), 27 June–1 July 2022, Milan, Italy.
• Juuso Haavisto, Thibault Cholez, Jukka Riekki, “Unleashing GPUs for Network Function Virtualization: an open architecture based on Vulkan and Kubernetes”, 18th IEEE/IFIP Network Operations and Management Symposium (NOMS 2022), Apr 2022, Budapest, Hungary, pp.9
• Zujany Salazar, Huu Nghia Nguyen, Wissam Mallouli, Ana R. Cavalli, and Edgardo Montes de Oca. 2021. "5Greplay: a 5G Network Traffic Fuzzer - Application to Attack Injection". In The 16th International Conference on Availability, Reliability and Security (ARES 2021). ACM, Article 106, 1–8. DOI:10.1145/3465481.3470079
• M. Letourneau, K. B. N’Djore, G. Doyen, B. Mathieu, R. Cogranne, H. N. Nguyen. 2021. “Assessing the Threats Targeting Low Latency Traffic: the Case of L4S”. In 3rd International Workshop on High-Precision, Predictable, and Low-Latency Networking (HiPNet). 17th International Conference on Network and Service Management (CNSM), Izmir, Turkey, 25-29 October 2021. DOI: 10.23919/CNSM52442.2021.9615534
• P. Graff, X. Marchal, T. Cholez, S. Tuffin, B. Mathieu, O. Festor,  ”An Analysis of Cloud Gaming Platforms Behavior under Different Network Constraints”. In 3rd International Workshop on High-Precision, Predictable, and Low-Latency Networking (HiPNet), . 17th International Conference on Network and Service Management (CNSM), Izmir, Turkey, 25-29 October 2021. DOI: 10.23919/CNSM52442.2021.9615562
• B. Mathieu, S. Tuffin, “Evaluating the L4S Architecture in Cellular Networks with a Programmable Switch” . In 26th IEEE Symposium on Computers and Communications (IEEE ISCC), 5-8 September 2021, Athens, Greece. DOI: 10.1109/ISCC53001.2021.9631539

Dissemination

• 12 July 2023: presented at 3rd Data Science workshop by European University of Technology (EUt+)
• 23 February 2023: presented at the 24nd ROADEF congress organized by French Society of Operational Research and Decision Support, "Heuristique de placement et de chainage de Services décomposées en micro-services" (see abstract and slides)
• 10 Novembre 2022: presented at Journée NetOpt 2022 Optimisation et algorithmes pour les réseaux Soutenue par les GdR RSD et l’axe REST du GdR-RO. "Placement et Chainage de Fonctions de Services composées de Micro-Services : Modélisation et Résolution" (see abstract and slides)
• 25 October 2022: presented online at the 7th Franco-Japanese Cybersecurity Workshop held at Keio University, Tokyo. "Featuring, Monitoring and Detecting novel attacks targeting Low Latency services: the case of L4S".
• Septembre 2022: presented at Orange Datacamp#2022, an annual event regrouping Orange people working on AI (# 200 this year) : (see slides here)
• June 2022: J. Ky, B. Mathieu, A. Lahmadi and R. Boutaba. Characterization and troubleshooting of cloud gaming applications on mobile networks. Network Traffic Measurement and Analysis Conference 2022 (TMA '22) PhD School, Enschede, The Netherlands, June 27-28, 2022. Recipient of the Best Poster Award. (See poster here)
• 31 March 2022: poster presented at IMT on the theme of "crisis management: new threats and new solutions" (see poster here)
• 15 March 2022: presented at 1st Workshop of the European Research Institute (ERI) on Telecommunication and Networks (see slides here)
• April 2021 : presented at the 22nd ROADEF congress organized by French Society of Operational Research and Decision Support. This edition was performed online (see slides here)
• May 2021 : presented at a conference on network security organized by the national research group SSLR (System Software and Network Security) and RSD (Networking and Distributed System) of the CNRS. (see paper here, poster here and slides here)
• December 2020: presented at the sixth edition of RESSI organized by the national research group. This edition was performed online. (see poster here)
• Novembre 2020: presented at the fifth edition of Cloud Days organized by the national research group RSD (Networking and Distributed System) of the CNRS. This edition was performed online (see slides here)

Dataset

Micro-services orchestration

Code in C++ for a heuristic that manages the placement and chaining of different Service Function Chains (SFC), leveraging the microservices approach under various constraints. Also, to minimize latency, the heuristic optimizes parallelization and load balancing during deployment (download code here).
 

Code in C++ simulating a micro-service orchestration model in an ultra-low latency context as well as the relative test instances. The code uses the IBM Cplex solver for its resolution. It allows the placement and chaining of different SFC leveraging the micro-services approach under different constraints.

• Variant considering only Low Latency (LL) SFC (download code here)

• Variant supporting cohabitation between Low Latency (LL) and Best Effort (BE) SFC (Paper in submission, please contact the authors to obtain the code)

Cloud gaming traces

The dataset hosted on this website are traces made during cloud gaming sessions. These captures were made under different network conditions.

• "Without settings": network captures under normal conditions.
• "With settings": network captures under disturbed conditions (using netem)
• "Emulating cellular networks": network captures under real cellular network conditions (using Mahimahi/Linkshell and captures of Orange 4G network : see below)

This dataset was used in the preparation of this publication : An Analysis of Cloud Gaming Platforms Behavior under Different Network Constraint and for a paper currently under submission.

Cloud gaming anomaly detection

This is the dataset used for the comparative evaluation of 05 unsupervised Machine Learning (ML) models used in the paper “Assessing Unsupervised Machine Learning solutions for Anomaly Detection in Cloud Gaming Sessions” accepted at the 4th International Workshop on High-Precision, Predictable and Low-Latency Networking (HiPNet). It contains 10 .csv files of 14 QoS/QoE features collected while playing “Dirt5” racing game on the Google Stadia cloud gaming platform under 6 different emulated 4G network conditions.

Link to the dataset here.
Link to the code used in the paper here.

 Cellular Network traces (to be used with Mahimahi / Linkshell tool)

The dataset hosted on this website allows to emulate the behavior of one base station deployed in Orange 4G network, in France.

6 network conditions are available : 5 static configurations based on the average downlink throughput and one of a car traveling at the steady speed of 110km/h on a highway.

This dataset was used in the preparation of a paper currently under submission.

 

Undesirable flow implementation using picoquic

Undesirable flows such as unresponsive ECN-capable flows, applicative bursts or flows without pacing are generated with an implementation based on the L4STeam fork of the privateoctopus' repository of picoquic, a minimalist implementation of QUIC. The fork from L4STeam is based on draft-17 of the protocol and is the version we used in our repository here.

Endpoint and DualPI² router traces are formatted in a custom .csv file containing metrics collected by tc (traffic control) or ss (socket statistics). One .csv file is generated for both low latency server and DualPI² router for each experiments (i.e. for 10/20/100 Mbps and for each type of undesirable flow).

The code for the adversarial Picoquic traffic as well as the related endpoint and DualPI² router traces are used in a paper currently under submission. During the submission process, to get a copy, please contact: Marius Letourneau, Guillaume Doyen, Rémi Cogranne.