Data Analytics developed by using Snap4City solutions and tools

×

Warning message

  • You can't delete this newsletter because it has not been sent to all its subscribers.
  • You can't delete this newsletter because it has not been sent to all its subscribers.

  

CLICK on the above thumbnail to get a booklet on data analytics of Snap4City, Snap4Industry cases

We suggest you to follow the training: https://www.snap4city.org/944

Older information and training as follows:

 

Early machine learning solutions, ML, were mainly black box generating scepticism over their applicability in critical situations in which one should be requested to just trust them. On the other hand, ethical aspects (on data and processes) are very relevant and a wrong assumption in taking data and/or setting up solutions may lead to biased results/suggestions, which may correspond discriminations and may lead to unforeseen costs. This was a real concern for former solutions and for the first generations of machine learning.

Snap4 technology is providing Big Data Analytic with Artificial Intelligence, AI, for leveraging businesses and solutions in producing reliable predictions, prescriptions, early warning, classifications, detections, suggestions, optimizations, and generation. In substance, Snap4 technologies lead to reduce costs and increase the efficiency of business and/or production processes. This implies to extract value from data, providing hints, prescriptions, suggestions, strategies, mitigations, and discovering solutions, information and implications never detected before, to learn and derive aspects of the phenomena up to now only observed and measured.

The applications can be in almost any domain of smart city and industry: mobility, health, energy, environment, waste, chemistry, manufactory, delivering, agriculture, boating, building, security, safety, etc.

The resulting advantages can be for final users, for decision-makers and for the city/companies. Examples can be:

  • monitoring and controlling traffic, making prediction, traffic flow reconstruction, simulating and performing what-if analysis for strategic plan minimizing traffic congestions, reducing emissions, etc.
  • Optimized estimation of traffic light plant taking into account public and private traffic, minimizing reducing: stops, travel time, and stops for tramways, waiting time, etc.
  • Optimized traffic infrastructure viability in terms of roads, directions, lanes, etc., reducing traffic congestion, travel time, emissions, fuel consumption, etc.
  • optimized positioning of bus stops, increasing the quality of transport experience and avoiding overcrowding, maximizing efficiency, minimizing costs, and decreasing the number of trips.
  • parking predictions to reduce the social costs of looking for parking: reduction of fuel consumption, reduction of produced as NO2/CO2 for the whole community, etc.
  • optimized routing based on present and predicted conditions, regarding traffic and maintenance works, for private drivers and city operators. Thus, for emergency services, reducing travel time, pollution, and the time required to reach the objective and perform services.
  • understanding city usage conditions, and city users’ satisfactory and demanded services, etc. Increasing quality of services by solving the identified problems via collective intelligence.
  • optimization of waste collection that reduces costs by minimizing the number of required trucks/trips and prevents waste from escaping bins through the use of predictive analytics. It is an advantage for the quality of life of city users and reduces administrative costs.
  • predicting maintenance (for roads and services) to reduce intervention costs, operating costs associated with unexpected faults and services/productions interruptions, avoiding damages provoked by faults, etc. This also implies a reduction of the costs of production improving efficiency and resilience.
  • sharing service prediction to reduce the time need to get/find a suitable sharing vehicle. Reduce costs and improve efficiency by means of sharing and pooling. Thus, increasing the quality of service also increases its effectiveness.
  • assessing and predicting reputations of services, attractions, based on social media, with the purpose of enhancing the quality of services, reducing prices increase, and promoting alternative offers and solutions.
  • early warning about possible landslides in the territory by computing predictions about landslides. Thus, reducing the risk for population, reducing the reaction time improving resilience of the territory.
  • preparing the city and/or the industry to be more resilient to unexpected unknown events, natural or man-made disasters, by integrating simulations and ML/AI solutions enabling the what-if analysis in near real-time, increase resilience and capacity. Rapidly reacting to unanticipated, unforeseen events, so reducing the cost of recovery, which is the risk, and receiving advice to mitigate the risks and damages.
  • Understanding the usage of city services to optimize energy and other resources needed for the expected serviceability and quality.
  • Dialoguing with decision makers by using natural language, understanding and producing suggestion, generating solutions via Generative AI such as LLM.
  • solving complex physical equations with Physically Informed Neural Networks, PINN, for Simulating fluid-dynamic flows in autoclaves and machineries, cost reduction, time reduction.

Snap4 AI/XAI solutions are capable to provide high precisions on predictions, classifications, prescriptions, suggestions and scenario generations and thus they can actually support decision makers as one of your best experts without humans’ bias. Snap4 AI/XAI solutions should be treated as a trusted expert to support decision makers to collaborate creating tailored solutions, and strategies to mitigate and solve short- and long-terms plan problems, as well as to support processes of What-If analysis, which were previously developed solely through simulations on the basis of hypothesis, and now the most effective solution can be directly generated by AI. Decision-makers can get suggestions shortening the boring activities and the AI can learn from the decision-makers’ objections improving progressively the AI capabilities and your preferences, taking into account additional aspects and elements. The recent solutions based on Large Language Models, LLM, may allow the decision maker to discuss with the expert in natural language. On the other hand, technical solutions still need to be generated by specialized AI rather than generalist LLM.

Extending these capabilities, the AI solution would allow entering in the loop with the community of users, and this would result in the production of new data for training (suggestions produced by the users to the AI, generative AI, as well as collective intelligence, from Participative solutions as Snap4City mobile Apps) which can be actually improve the AI precision and capability in modelling the phenomena and providing effective suggestions. Some of these approaches fit in the reinforced learning, transfer learning, fine tuning, and in the continuous learning techniques. Moreover, a relevant push has been provided by semantic reasoner tools (such as Km4City since 2013, in Snap4City), which started with the definition of ontologies and collecting data become actual expert systems which can be queries with semantic query language to perform inference and gest smart suggestions and results, neuro symbolic solutions. Examples are the graph neural network, graph data stores and the spatial, temporal and entities reasoners.

 

In recent years, AI has rapidly evolved and started to be used in complex systems and not only to produce direct predictions and prescriptions. Thus, decision-makers started to use the new solutions expecting to see them respecting the ethics (on data and processes), with aim of trusting them as their best experts. To this end, AI trustworthy, Data Ethics and AI Ethics approaches have been created by international and national bodies to guide AI developers in producing solutions which can actually serve the decision makers without prejudice and with correctness. Data Ethics refers to the aspects that may provoke a bias and ethical problems since the training; for example, training the AI with biased data, unbalanced distribution of cases, etc. Moreover, specific AI methodologies and solutions for Explainable Artificial Intelligence, XAI, are presently providing support in this direction since they are capable to explain the rationales behind the typical results provided (global explainable AI) and may provide specific description/rational for each result/suggestion provided (local explainable AI). XAI typically adds value to the suggested decisions producing hints and discovering implications and correlations never detected before by humans. They are a source of information to train the decision-makers about what has been discovered to be relevant for the AI. On the other hand, the decision makers can also provide to the AI continuous and lifelong training inputs for improving the capabilities of the AI. Thus, in Europe (April 2019) and in recently in other countries normative are going to set up regarding the AI Ethics, proving guidelines on training AI for Ethics, thus for AI trustworthy, Data Ethics and AI Ethics.

Read more on: https://www.snap4city.org/download/video/course/p4/

Snap4City Analytics, the data analytics at your disposal on Snap4City

Snap4City has been designed since the 2017 to be AI enabled, respecting ethics, secure passing the PENtest and GDPR compliant. Snap4City has developed a large number of solutions in the context of Smart City and Industry 4.0. Snap4City fully supports the development of real time data analytic processes through ML, AI, ethic trustworthy, XAI via languages such as Python, R-Studio, also exploiting Tensor Flow, Pandas, Keras, BERT, LLM, and any kind of library for data analytics, ML and AI. Snap4City is distributing a number of Open-Source data analytics tools and algorithms for: prediction, anomaly detection, classification, detection, constrained routing, optimization, analysis of demand vs offers of transportation, and many others have been published on international top level journals and can be customized on demand on your cases. Artificial Intelligence and data analytics support are fully integrated into What-If analysis and optimisation tools in control rooms and for the operators, defining scenarios and solutions. Snap4City has a consolidated experience in the development, validation and transfer AI/XAI solutions (see course https://www.snap4city.org/944). Most of the DISIT lab solutions are based on ML, Deep learning, AI, XAI, natural language processing (NLP), sentiment analysis (SA), semantic reasoning and computing, neuro symbolic, generative AI, reinforced learning, etc. In the following, a number of examples are listed, while more details can be recovered from the Snap4City course and from technical notes: https://www.snap4city.org/4  and a number of AI solution is accessible: https://www.snap4city.org/997

Mobility and Transport Domain

  • Goals:
    • Decongestion, Decarbonization, Accessibility to services
    • Security/Safety of city users
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring traffic, parking, people flow, services, boats, ports, beaches, etc.
    • Early detection/warning of critical conditions: traffic, congestion, security/safety
    • Managing Smart Parking, transportation services, fines, etc.
    • Managing fleets: personal, sharing, waste collection, maintenance, etc.
    • Managing E-sharing, pooling services, MaaS, etc.
    • Managing entrances in city areas: restricted areas, touristic busses, etc.
    • Production of suggestions, recommendations, nudging 
    • Computing predictions of any kind
  • Solutions for Planning (optimization and what-if analysis)
    • Reduction of traffic congestion, via optimization: traffic light plans, viability, routing
    • Reduction of Pollutant Emissions, via optimization: traffic light plans, viability
    • Optimization of transportation offers wrt multimodal mobility demand
  • Algorithms and computational solutions
    • Optimisation of viability of an area for reducing congestion, waiting time, stops
    • Optimisation of Traffic Light Plans, synchronization, in an area for reducing congestion, waiting time, stops
    • Predictions for: traffic flow, smart parking, smart bike sharing, people flows, etc. (ML, DL)
    • What if analysis: routing, traffic flow, demand vs offer, pollutant, etc. (Simulation + ML)
    • Traffic flow reconstruction from sensors and other sources (simulation + ML)
    • Public Transportation: Ingestion and modelling of GTFS, Transmodel, NeTEx, etc. (DP)
      • Analysis of the demand mobility vs offer transport of according to public transportation and multiple data sources (Simulation)
      • Assessing quality of public transportation (analysis)
    • Accidents heatmaps, anomaly detection (analysis, ML)
    • Road light controlled by traffic conditions
    • Tracking fleets, people, via devices: OBU, OBD2, mobile apps, etc. (DP)
    • Routing and multimodal routing (multistop travel planning), constrained routing, dynamic routing (DA)
    • Computing Origin Destination Matrices from different kind of data (analysis, DP, DP)
    • Computing typical trajectories on the basis of tracks (analysis, ML)
    • Fleet management, monitoring, booking, allocation, maintenance
    • Computing Messages for Connected drive (DP)
    • Slow and Fast Mobility 15 Minute City Indexes (analysis, DP, …ML)
    • Computing and comparing traffic flow on devices and at the city border (analysis)
    • Typical time trends for traffic flow and IoT Time series. (analysis, ML)
    • Impact of COVID-19 on mobility and transport
    • Computing SUMI, PUMS, etc. (mainly DP)
    • Definition of Scenariostraffic, road graph, conditions, etc.

City User Behaviour/services, Tourism and Safety

  • Goals:
    • Quality of Life, quality of services, over tourism mitigation, sustainability
    • Costs reduction of services
    • Accessibility to services: citizens, Tourists, commuters, etc. 
    • Security/Safety of city users
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring services: tickets, reputation, usages, areas, etc.
    • Monitoring user behaviour (counting, trajectories): indoor/outdoor, hot places/services, ports, beaches,
    • Computing: origin destination, trajectories, travel means, reputation, predictions, etc.
    • Early detection/warning of critical conditions, connection with Video Management Systems
    • Managing entrances in city areas: restricted areas, touristic busses, etc.
    • Production of info-tourism, recommendations, nudging to city users and operators, second offer promotion
    • Providing Virtual Assistants for City Services, Tourist Offices, etc.
    • Monitoring reputation of services via: social media, blogs, etc.
    • Collecting complains, requests, participations from City users via mobile apps
    • Computing predictions of any kind: people coming/moving, services and sites reputation, advertising impact and people reactions.
  • Solutions for Planning (optimization and what-if analysis)
    • prediction of the effect of certain changes on the offer;
    • Reduction of Pollutant Emissions, via optimization
    • Optimization plan to distribution of workload on multiple touristic offers/services, area cleaning, etc.
    • Predicting reputation of services, touristic and operative
  • Algorithms and computational solutions
    • People detection and classification: persona, strollers, bikes, etc. (ML, DL)
    • people counting and tracking, head counting, people trajectories (via thermal cameras, ML, DL)
    • People flows prediction and reconstruction, (ML, DL)
    • User’s behaviour analysis, People flow analysis from PAX Counters and heterogenous data sources (ML, AI)
      • origin destination matrices, hot places, time schedule,
      • Recency and frequency, permanence, typical trajectory, etc.
    • Computing User engagement and suggestions for sustainable mobility (Rule Based, ML)
    • Social media analysis on specific channel, specific keywords: see Twitter Vigilance,
      • Reputation, service assessment: MultiLingual NLP and Sentiment Analysis, SA
      • Tweet proneness, retweet-ability of tweets, impact guessing
      • Audience predictions on TV channels and physical events, locations
      • Prediction of attendance of events and on attractions
    • Virtual Assistant construction, LLM, NLP, Sentiment Analysis (DL, NLP)
    • Video management System integration for security
    • 15 Minute City Index, etc. (modelling and computability)
    • Computing SDG, etc., (DP)

Environment, waste, land, etc., Domain

  • Goals:
    • Reduction of emissions and EC taxations
    • Cost reduction for waste collection, reduction of waste collection impact on mobility
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring emissions, weather, waste, water, etc.: sensors, traffic, flows, ….
    • Early detection/warning of critical conditions on emissions, weather, waste, water, fire, animals, …
    • Early detection/warning of critical conditions for landslides, water flooding, beach
    • Smart Waste Management: bins/lockers, waste collection daily plan, pay as you throw, PAYT, etc.
    • Short terms prediction of emissions: CO2, NO2, etc.
    • Production of suggestions, nudging 
    • Computing and predicting of long terms KPI indicators of the European Commission
  • Solutions for Planning (optimization and what-if analysis)
    • Identification of main CO2/NO2 emissions locations in the city, total production from traffic
    • Reduction of Pollutant Emissions, via optimization: semaphore cycles, viability
  • Algorithms and computational solutions
    • Pollutant Predictions: short, long and very long term European Commission KPIs
      • NOX, PM10, PM2.5 pollution on the basis of traffic flow, 48 hours (ML, AI, DL)
      • Cumulated NO2 average over year (ML, AI, DL)
    • Computation of CO2 on the basis of traffic flows (DP), computing emission factor (DA)
      • each road for each time slot of the day
    • Prediction of MicroClimate conditions for diffusion (ML, AI)
      • NO2, PM10, PM2.5, etc.
    • Prediction of landslides, 24 hours in advance (AI, DL)
    • prediction of waste collection, & optimisation of schedule and paths (DP, ML)
    • Heatmaps production dense data interpolation (DP) for
      • Weather conditions: temperature, humidity, wind, DEW
      • Pollutants and Aerosol: NO, NO2, CO2, PM10, PM2.5, etc.
    • Impact of COVID-19 on Environmental aspects (DP)
    • Computing SDG, SUMI, SUMP, .. (mainly DP)

Snap4Building Domain

  • Goals:
    • increase efficiency, cost reduction, sustainability
    • Accessibility to services, Security/Safety
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring: usage, energy, environmental conditions, people flows, services, etc.
    • Early detection/warning, alarm, of critical conditions, notifications, decision support
    • Production of suggestions/prescriptions, nudging 
    • Managing smart services: cabinets, dispenser, lockers, etc.
    • Global and local 3D/2D representations of area and buildings
    • Integration with Video Management Systems
    • Computing predictions of any kind
  • Solutions for Planning (optimization and what-if analysis)
    • Reduction of energy costs via optimization
  • Algorithms and computational solutions
    • Digital Twin for monitor, control and manage distributed infrastructures
      • 2D/3D representations of the whole set of buildings, BIM modelling
      • Entities (building, floors, rooms, parking, charging stations, gates, etc.) with their shapes and descriptors, and data monitoring the allocation to office, meeting, cafeteria, storage, stairs, elevator, etc.
    • Monitoring and computing KPIs on real time for
      • energy consumed or produced (hot/cold)parking, logistic, presences, cleaning, air quality, departments, subareas, maintenance, etc.
      • allocation/designation, dispositions, heating, cooling, temperature, equipment, etc.
      • grouped in Zones

Energy Domain

  • Goals:
    • Energy consumption reduction, increment of efficiency, sustainability
    • accessibility to services
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring energy consumption (heating, cooling, prod.,..), conditions, charging stations, etc.
    • Managing Smart Light for city: dimering, programming, traffic control, controllers, legacy, etc.
    • Early detection/warning, alarm, of critical conditions
    • Managing smart services: cabinets, lockers, etc.
    • Production of suggestions, nudging 
    • Global and local 3D/2D representations of area and buildings
    • Managing Communities of Energy, certification via Blockchain
    • Computing predictions of any kind
  • Solutions for Planning (optimization and what-if analysis)
    • Reduction of energy costs, via optimization
    • Identification of roofs with better orientation
    • Optimization of battery storage size for PV plants
    • Community of Energy planning and viability
  • Algorithms and computational solutions
    • Monitoring Energy Consumption in single building, area and per zone
    • Smart Light management, unicast and multi cast management, smart light controlled by traffic flow data
    • Monitoring Energy provisioning on recharging station
    • Matching Energy consumption with respect to the actual usage
    • Computing Roof orientation for Photovoltaic installations
    • Optimisation of Photovoltaic installations to identify the best parameters of size and storage
    • Collecting and managing Communities of Energy
    • Computing KPI
    • Etc.

Assets Control Domain

  • Goals:
    • Costs reduction, increase service availability, risk reduction, Quality Level
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring :
      • Assets: switches, Wi-Fi, servers, UPS, sensors, building, TV Cams, etc.
      • Energy: consumption, operative conditions, etc.
      • Production: continuous quality analysis
      • Etc.
    • Early detection/warning, alarm, of critical conditions
      • Multichannel Event reporting: email, Telegram, mobile apps, SMS, etc.
    • Managing maintenance operation
    • Computing predictions of any kind
  • Solutions for Planning (optimization and what-if analysis)
    • Reduction maintenance costs, reduction of critical SLA conditions, improvement of quality level

Industry production Domain

  • Goals:
    • Cost reduction, increase control on production, Production optimisation
    • Quality Level
  • Solutions for Operation (monitoring, managing, mobile apps, digital signages, control rooms)
    • Monitoring KPI: administration, production, commercial, faults, etc.
    • Early detection/warning, alarm, of critical conditions
      • Multichannel Event reporting: email, Telegram, mobile apps, SMS, etc.
    • Managing maintenance operation, predictive maintenance
    • Computing predictions on KPI
    • Computing predictive maintenance
    • generation of patterns in production, design, etc.
    • solving complex physical equations with Physically Informed Neural Networks, PINN, for Simulating fluid-dynamic flows in autoclaves and machineries, cost reduction, time reduction.
  • Solutions for Planning (optimization and what-if analysis)
    • Generative AI and predictive AI for production plan optimisation
    • Reduction maintenance costs, reduction of critical SLA conditions, improving quality level

They are developed by using a large range of statistics, operatig research, ML, AI, XAI techniques, such as those classified as Deep Learning, GNN, Generative AI, Reinforced Learning, KB reasoning, neurosymbolic reasoners.

The usage of Snap4City has brought about improvements and has been of great benefit to a wide range of applications where it has been adopted for operational management, prediction and plans. The very low costs for its adoption and usage, has impressed many adopters, which may also add functionalities and perform changes by exploiting the visual tools. Adopter may delegate the maintenance to Snap4City of may take full control of the platform with limited effort.

Its capability and compliance at European level allowed to perform a huge number of integrations and particular in the cities of Merano, Cuneo, Valencia, Rhodes, Antwerp, Pisa, Malta, Livorno, Firenze, Modena, Santiago de Compostela, Pont du Gard, Dubrovnik, Lonato del Garda, Helsinki, Bisevo, Mostar, Varna, etc. And it has been adopted in a number of European, national and regional projects: REPLICATE lighthouse H2020, RESOLUTE H2020, TRAFAIR CEF, Sii-Mobility MIUR, SODA4.0 of ALTAIR, 5G MIUR, MOBIMART Interreg, HERIT-DATA Interreg lighthouse, Life Weee, IMPETUS, MOSAIC, AMPERE, Enterprise, PANACEA, Pretto, ALMAFLUIDA, and PC4City. These actions have involved a large number of partners from private industries and public institutions (cities, regions, universities, foundations) working and using Snap4City platform.

And more recently, since 2024, as the reference platform for: TOURISMO Interreg, AMMIRARE Interreg, Tuscany X.0 EDIH, ELLIS Horizon Europe, SASUAM scalability of CN MOST (Spoke 8), OPTIFaaS Flagship action of CN MOST (Spoke 8 and 9), CN MOST (national centre on sustainable mobility in Italy, Snap4city is one of the reference platform for Spoke 9), and CAI4DSA of FAIR PE (national project on Artificial Intelligence for society). For most of them, a dedicated web page is provided on main platform https://www.snap4city.org/135 Moreover, since a number of years Snap4City platform is on progressive adoption of the SOC of ISPRA JRC of the European Commission.

Most of this strong ride started since 2013 with the first data integration for Florence city, and in the 2019, in which DISIT Lab (University of Florence) turned out to be the winner of the Select4Cities PCP of EU managed by Antwerp, Helsinki and Copenhagen, one year later won the ENEL-X open data challenge in 2020. Also, Herit-Data action adopted the Snap4City platform for all pilots and received the LightHouse flag from the European Commission. Currently, Snap4City is one of the platforms of the EOSC (European Open Science Cloud), library of Node-RED, and DISIT Lab is proud to be a Gold Member of FIWARE and an official FIWARE Platform and Solution, certified Consultant, certified Trainer, provides two certified FIWARE Experts; and recent best awards for Digital Twin platform for Smart Cities from DMS, ICCSA.

-------------former versions-----------------

  1. First step to reduce the fossil emission is to monitor the present emission, identify the most critical points and factors. The emissions from fossil combustion are mainly CO2 and NO1, NO2. And they are due to vehicles, house heating and industries… but where they are now precisely in the city and lands
  1. As first step the present traffic and housing conditions have to be assed, then the progresses can be kept under control. The assessment can be performed by computing indicators. For example the 15 Min City index for serviceability at 15 minute distance by walking in the city and with different parameters in peripheral and rural areas, which includes an index for housing and slow and fast mobility. Relevant aspects are: (i) to assess the capability of  the public transport to satisfy the demand, (ii) predicting traffic flow and reconstruction, (iii) making the what if analysis to be capable to assess how the traffic conditions would change due to changes in the road network, for example for restructuring, etc.

--- former version---------

This page reports what Snap4City has done for Helsinki and Antwerp in terms of Data Analytic.

List of all scenarious: https://www.snap4city.org/4

Please note that Snap4City has also other capabilities and tools on this matter so that see also:

 

All the algorithms have worked continuously providing services to the users and also information to US: What they requested, What they visited, POI; where they passes: trajectories and OD matrix, Language, Routing, Clicking on services and icons, Engaging: responding, Searching.

 

Description of the main categories of Data Analysis provided.