Tag Archives: artificial intelligence

Interview with 2024 Research Showcase “Outstanding University Student in Transportation Research Award” Winner

Posted on

Traffic safety and mobility, two critical areas in transportation engineering, both require the collection and analysis of large data sets to produce proactive and comprehensive solutions. Transportation engineers have started to increasingly focus on using innovative technologies to efficiently and effectively process this data.

We had the opportunity to speak with Dr. Deep Patel, a former Ph.D. candidate and research fellow at Rowan University, whose work is at the forefront of this mission. Recently, Patel received the NJDOT Outstanding University Student Research Award for his contributions to transportation research. In this interview, Patel shares insights from his research journey, including his work on the Real-Time Traffic Signal Performance Measurement Study and the development and implementation of machine learning tools to predict high-risk intersections. His dedication to improving traffic operations and safety, along with his new industry role as a Traffic Safety and Mobility Specialist, highlights the significant impact of combining academic research with practical industry applications.

Q. Could you tell us about your educational and research experience and how you became a PhD candidate and research fellow at Rowan University?

A. First of all, thank you for your time and for considering me for the opportunity to be interviewed about my recent NJDOT award. I would also like to thank the NJDOT review committee members and my Ph.D. advisor Dr. Mohammad Jalayer, who supported me in receiving this award.

I started my master’s study in 2018 as a civil engineering student without a research focus. Then, during my first semester, I took a course called Transportation Engineering with Dr. Mohammad Jalayer. When he sought traffic counting assistance for a traffic analysis project, I eagerly joined him, becoming his first research student.

Deep Patel conducting roadside research. Courtesy of Deep Patel.

Through that experience, I started thinking about what could streamline the traffic counting process and the various uses for the data we collected. I went on to work on several research projects with Dr. Jalayer, both funded and non-funded, where we had frequent discussions, and I would present my ideas to him. Eventually, he asked me to join him as a researcher and to continue my master’s work with a research focus, which I did for two years. When he suggested I continue my studies to earn a Ph.D., I was initially surprised, but I decided to go for it since I had a lot of ideas for future research projects.

At the end of my master’s study, I began Phase One work for a Real-Time Traffic System Performance Measure Study led by Dr. Peter Jin, Dr. Thomas Brennan, and Dr. Jalayer. This project connected me with a team from Rutgers, TCNJ, and a few professionals from NJDOT and other industry folks. I represented Rowan’s end for this project, where our focus was on understanding the safety aspects including safety parameters and performance and how we could assist NJDOT transform this new technology to help save lives. For the first phase of the project, we worked on understanding the traffic signal system performance measures, and what had been adopted by other DOTs. My experience on this project drove me to pursue more research and to expand my knowledge in traffic safety.

Q. You worked on Phase One through Three of this Real-Time Traffic Signal Performance Measurement Study. What part of this project interested you the most?

A. My main takeaway from this project focused on learning more about how the transportation industry looks towards the research outputs and outcomes from the university teams. It is very interesting to understand how university-based research is being adapted for industry acceptance. Additionally, I learned what problem-solving features the industry looks for from the research component.

From a technical aspect, I learned how New Jersey signals can be enhanced and how we can optimize the performance of these signals and achieve cost savings. Let’s say you have a scenario where there is no vehicle at an intersection; how can we provide recommendations to change the signal to a red light and give the other side of the intersection a green light? So, we gathered several components in terms of mobility, safety, and economic parameters from the study that can help enhance our traffic signals in New Jersey, sharing this information with the NJDOT team.

Figure 1: An Example real-time performance monitoring on County Road 541 and Irwick Road, Burlington County, NJ
Example of real-time performance monitoring on County Road 541 and Irwick Road, Burlington County, NJ

Q. How did you see your role on the research project develop as you moved from the earlier phases to the latest phase?

A. In the first phase, we completed a comprehensive literature review to understand what is happening across the nation, which systems are being adapted, what are the best systems for providing traffic signal safety performance measures, and what are the kind of performance measures that can be adapted in an industry setting. In Phase Two, the team focused on developing mechanisms and performance measures aligned with NJDOT’s existing data, including deploying the Automated Traffic Signal Performance Measures (ATSPM) system to enhance traffic signal monitoring and optimization. To guide these efforts, an adaptability checklist was created to benchmark practices from other states and identify strategies that could be adapted to benefit NJDOT’s operations. Building on this foundation, Phase Three advanced to the demonstration and application of dashboards and performance measures, providing actionable recommendations to NJDOT on enhancing mobility and safety across various regions and corridors. These efforts aimed to save time and lives, while the integration of connected vehicle (CV) technologies remains a key focus for future work, ensuring NJDOT’s leadership in traffic management innovation.

Q. What were the specific corridors that you worked on?

A. We started with seven/eight intersections on U.S. 1. Then, we explored the whole corridor of U.S. 1 as part of Phase Three, and we also brought in Route 18, Route 130, and other intersections during this phase.

Q. Did you discover any particular surprising or noteworthy findings from this research?

A. This was a long project, extending from 2019-2024. As a result, each year we discovered new findings, and new components were often added to the project. For example, we added a CV systems component as part of the Phase Two and Phase Three projects to start planning for the future and understand what kind of data could be received and sent from CV technologies. The main benefit from this project is that it not only established current problem-solving measures but also looked into the future, helping to better understand what’s coming and how we can best face anticipated challenges that we need to start integrating at this moment. I find the combination of the present and future integration of systems and technologies interesting and important from the findings.

Q. What kind of impact do you think you and your research will have on NJDOT traffic operations and traffic safety, especially with your role now working in the industry?

A. With my previous experience as part of a university-led research team and now as a Traffic Safety and Specialist in the private sector, I am better positioned to facilitate the efficient and effective implementation of research findings.  A key factor enabling this transition is that Kelly McVeigh, who supervised the original research project, also oversees the current work that our firm is doing for NJDOT. Being on the industry side allows me to introduce and operationalize new ideas more rapidly, compared to the academic research side. This streamlined approach ensures that innovative performance measures can be deployed more quickly, and even a small modification has the potential to save lives, underscoring the value of this work.

Q. Moving to a different topic, your research frequently incorporates Machine Learning (ML) and Artificial Intelligence (AI) aspects. In your experience, what benefits does AI contribute to transportation research?

A. Over the past few years, AI and ML have undergone drastic modifications and growing levels of industry acceptance. Additionally, in research outcomes, AI and ML have played a key role in enhancing and providing new methodologies and new ways of problem-solving. As an engineer, the first thing we have to do is understand how we can solve an existing problem, and how fast, effectively, and efficiently we can do it.

Transportation is now highly reliant on big data and intensive analysis, so AI and ML back up the processing of this data, coming up with meaningful outcomes and enhancing solution measures much quickly than previous methods. In 2012 or 2013, a standard engineer would need to sit down to do a traffic study and go through manual counting, then process the data, then come up with solutions, which takes much longer to solve a problem. The problem may even change during the months-long process of developing a solution.

In traffic safety, we cannot wait for the four to five months it could take to solve a problem due to the pressing safety implications of doing so. Thus, we must start implementing countermeasures swiftly, and AI and ML components help us to quickly process data with more effective and efficient results.

During my early days as a student researcher, I would stand on the roadside, manually counting the vehicles and pedestrians to collect data for traffic studies. However. during my doctoral research, I developed my AI-driven tools that utilize advanced video systems for detection and analysis. This proactive approach enables the identification of intersections prone to high-crash scenarios well before crashes occur, allowing for timely interventions. By integrating AI and ML, my research introduced innovative methodologies for crash prediction and prevention, showcasing the feasibility of data-driven solutions to enhance roadway safety.

There is a certain chaos in human beings’ lives and surroundings that requires transportation to be a multidisciplinary field, which includes human-focused aspects. For some parts, AI is definitely required, but with other parts, we need to go through different approaches.

Q. Do you think that because of AI’s data collection and analysis possibilities, almost all engineers in the near future will need to start incorporating AI into their research?

A. It really depends. For our part of traffic engineering, very specifically, I would say yes, it would be one of the major requirements that an engineer would need to adopt. But if I was a traffic engineer working on policy or equity measures there might be some concerns related to data sharing or data privacy issues that might restrict them.

It depends on what side you are focusing on. When it comes to data collection, I would say AI incorporation is a must to collect and process data faster and more efficiently. But in terms of developing policies, rules, or statutes, there are certain psychological aspects that need to be in the thought process. Knowing human concerns and people’s approaches requires an emotional touch, which AI still lacks.

Transportation is a field connected with multiple disciplines; it touches on people’s emotions. For example, on a day when traffic does not work well when you’re returning home, you can get frustrated, and that frustration can end up in a fatal crash. There is a certain chaos in human beings’ lives and surroundings that requires transportation to be a multidisciplinary field, which includes human-focused aspects. For some parts, AI is definitely required, but with other parts, we need to go through different approaches.

Q. Congratulations on your recently approved dissertation. Could you give us some quick highlights of the research methods that went into producing your dissertation, “A Comprehensive ML and AI Framework for Intersection Safety”? What are the most important takeaways from your dissertation?

Deep Patel presenting his poster at the 2022 NJDOT Research Showcase Poster Session. Click image for PDF of the poster.

A. New Jersey is home to some of the most dangerous intersections in the United States, with four intersections ranked among the top 15 most dangerous, including the 1st, 2nd, and 3rd positions. Since 2019, there has been a trend of steadily increasing intersection-related crashes and correlated crashes within intersection boundaries. This prompted me to ask, “Why do we need to wait for crashes to happen to address the problem?”

To tackle this issue, I developed a proactive approach inspired by my work on the NJDOT research project. The approach focuses on analyzing near-miss incidents and traffic violations, using the concept of surrogate safety measures to identify potential risks before crashes occur. Surrogate safety measures help us detect near-miss events and violations, offering a predictive understanding of high-risk scenarios at intersections.

Using AI and ML, we developed tools that analyze vehicle and pedestrian trajectories in detail. These tools detect and classify conflicts, such as left-turn conflicts or yielding conflicts, enabling us to predict potential crash scenarios based on behavioral patterns at intersections. This proactive analysis allows us to recommend design changes and interventions before crashes occur.

Then, we explored the noncompliance component in a certain area, like red light violations or jaywalking. For instance, our analysis revealed that one in every four pedestrians does not use crosswalks. By integrating historical crash data, proactive trajectory analysis, and noncompliance trends, we developed a tool that ranks intersections based on multiple criteria. These include potential high-crash scenarios, contributing factors, and the economic impact of injury severity at specific locations.

Determining Key Factors Linked to Injury Severity in Intersection-Related Crashes in NJ. Deep Patel, Rowan University (2023 Research Showcase). Click image for slides.

Additionally, the research explored how emerging technologies, such as connected and autonomous vehicles, could be adapted to enhance intersection safety. By conducting trajectory analyses, we assessed how data from these technologies could inform future safety measures and interventions.

Overall, my research focused on identifying key factors within intersection boundaries to reduce crashes, improve mobility, and do so in a cost-effective manner. This comprehensive approach combines proactive analysis, advanced technologies, and human behavior insights to deliver practical and impactful solutions for roadway safety.

Q. So this tool seems to be one of the most important takeaways. Is the tool ready for NJDOT use to identify potential high crash risk intersections? Is that the main intent of the tool?

A. Yes, exactly. The tool is ready but not yet publicly available. We tested it on several intersections. It is currently a proprietary tool of my professor and myself at Rowan University. Anyone interested in using the tool can connect with us, but it is not yet publicly available and certain permissions are required.

Q. Is NJDOT using it or can they use it?

A. No, the department is not using it because this was part of my recent defense. They are aware of the tool’s capabilities because it was part of an innovative showcase. The tool’s documentation has been published through the University Transportation Center (UTC). Hopefully, in the near future, it could be applied by NJDOT.

Q. Looking ahead, you have your new position in an industry role. Would you like to continue with this sort of focus on transportation research, or are you anticipating a different career direction?

A. With my new position as a Traffic Safety and Mobility Specialist, I will be focused on transportation research, conducting high-quality industry research where I would help develop safety and mobility performance measures on certain corridors designed to move traffic more effectively and enhance safety on the roadways. My work will also include industry deployment and understanding the agencies’ concerns regarding the challenges they face.

Looking ahead, I see my career direction as a blend of research and practical implementation, ensuring that innovative solutions are not just developed but also applied to make a real-world impact. Ultimately, if my work can contribute to saving even a single life, I will consider it a meaningful and worthwhile achievement.

Resources

Jin, P. J., Zhang, T., Brennan Jr, T. M., & Jalayer, M. (2019). Real-Time Signal Performance Measurement (RT-SPM) (No. FHWA NJ-2019-002).  Retrieved at: https://www.njdottechtransfer.net/wp-content/uploads/2020/01/FHWA-NJ-2019-002.pdf

Jin, P. J., Zhang, T., Brennan Jr, T. M., & Jalayer, M. (2019). Real-Time Signal Performance Measurement Phase II. Retrieved at:  https://www.njdottechtransfer.net/wp-content/uploads/2022/08/FHWA-NJ-2022-002-Volume-I-.pdf

Patel, D., P. Hosseini, and M. Jalayer. (2024). A framework for proactive safety evaluation of intersection using surrogate safety measures and non-compliance behavior. Accident Analysis & Prevention, Vol. 192. https://trid.trb.org/View/2242428

Patel, D. (2024). “A Comprehensive ML and AI Framework for Intersection Safety: Assessing Contributing Factors, Surrogate Safety Measures, Non-Compliance Behaviors, and Cost-Inclusive Methodology.” Theses and Dissertations. 3305. https://rdw.rowan.edu/etd/3305

For more information about the 26th annual NJDOT Research Showcase, visit: Recap: 26th Annual NJDOT Research Showcase

Did You Know? AI in Transportation

Posted on

Artificial Intelligence (AI) is rapidly reshaping transportation by improving safety, efficiency, and sustainability across various applications. From real-time traffic monitoring to predictive infrastructure maintenance, AI is becoming a critical tool for advancing transportation systems in New Jersey and nationwide. This article covers the use of AI in transportation research and implementation, with examples from the 2024 NJDOT Research Showcase, New Jersey and other state DOTs.  

AI on Display at the 2024 Research Showcase  

NJDOT held its 2024 Research Showcase on October 23, highlighting innovative transportation research and its implementation throughout New Jersey. During the morning panel discussion, Giri Venkiteela, Innovation Officer in NJDOT’s Bureau of Research, Innovation & Information Transfer, stated that Artificial Intelligence (AI) held significant promise for producing economic and environmental advancements in transportation due to its real-time predictive capabilities and proposed that NJDOT adopt protocols that can adapt to the pace of AI. Similar insights were heard throughout the showcase, where AI emerged as a central theme across numerous presentations and discussions.

AI, encompassing subcategories like Machine Learning (ML) and Artificial Neural Networks (ANN), allows researchers to analyze and model large data sets in real-time, saving significant labor hours and producing efficient, immediate results. Throughout the showcase, various projects ranging from enhancing pedestrian safety to predicting natural disasters utilized AI-based models.  

Deep Patel received the 2024 Outstanding University Student in Transportation Research. As part of a research team at Rowan University, Patel deployed the AI model, YOLO-v5, to analyze video data from multiple New Jersey intersections, providing information on pedestrian volumes, traffic volumes, and the rate of vehicles running red lights, among other variables. The team then ranked intersection safety using the metrics analyzed by the AI model. 

Slide from Meiyin Liu’s presentation on real-time traffic flow analysis.

Patel’s research exemplifies the growing trend of integrating AI methods into traffic safety analyses, which continued into several presentations given in the afternoon Safety Breakout Sessions. Here, Rutgers professor Meiyin Liu presented her method for estimating real-time traffic flow through a combination of Unmanned Aerial Systems (UAS) and deep learning algorithms. A computer-mounted UAS would be used to record video data of a highway, which then gets transmitted to the YOLO-v5 computer vision AI that detects vehicle volume and estimates speed. This data collection method facilitates a real-time traffic flow analysis across a comprehensive geographic coverage that could enhance traffic performance and crash risk prediction. Afterward, Branislav Dimitrijevic, a member of an NJIT research team, showcased an AI-driven project that utilized LiDAR technology and YOLO-v5 computer vision to activate a Rectangular Rapid Flashing Beacon (RRFB) when pedestrians approached crosswalks, enhancing road safety.

Poster by Indira Prasad from the 2024 NJDOT Research Showcase.

Multiple posters featured at the Research Showcase contained elements of AI, including a poster titled “Integrating AI to Mitigate Climate Change in Transportation Infrastructure” made by Indira Prasad and “Artificial Intelligence Aided Railroad-Grade Crossing Vehicular Stop on Track Detection and Case Studies” highlighted by researchers at Rutgers’ CAIT. 

AI’s critical role in the maintenance and preservation of infrastructure was also evident in the afternoon’s Sustainability Breakout Sessions. Indira Prasad, a Stevens Institute of Technology graduate student, conducted a review of future innovations in sustainable and resilient infrastructure. Prasad explained how AI’s pattern recognition capabilities could be used to analyze large data pools and help forecast natural disasters, enabling a rapid response to augment existing infrastructure. Surya Teja Swarna, a Rowan University postdoctoral researcher, demonstrated an innovative approach where state DOTs could use mobile phones mounted on vehicles to record roadway surface deformations, which then would be analyzed in real-time by an AI computer vision software, drastically reducing the time and costs required for road condition assessments.

Deployment of AI in Programs and Project Implementation  

In addition to research from academic institutions, State DOTs and various other state, local and public transportation organizations have started to deploy AI-based methods and tools on various programs and projects. 

Peter Jin, a Rutgers professor, received the 2024 NJDOT Research Implementation Award for his role in the New Brunswick Innovation Hub Smart Mobility Testing Ground (Data City SMTG).  The project, created in partnership with NJDOT, the City of New Brunswick, and Middlesex County, functioned as a living laboratory for transportation data collection, containing Self-Driving Grade LiDAR sensors and computing devices across a 2.4-mile multi-modal corridor. Private and public sectors can use the data to enhance their advanced driving systems, automated vehicle models, and other AI-based projects. 

Additionally, NJDOT has established a program integrating unmanned aerial systems (UAS) into its transportation operations. UASs provide high-quality survey and data mapping information, which, when paired with AI-based technologies, can be analyzed in real time to document roadway characteristics or conduct damage assessments for natural disasters. Meiyin Liu’s real-time traffic flow assessment research is one example of how UAS can be paired with AI. 

The methods used by CAIT to detect and analyze railroad-grade crossings. Courtesy of CAIT.

The use of AI for railroad-grade crossing detection has been demonstrated on several projects in recent years.  NJ TRANSIT, the statewide transit agency, recently received a $1.6 million grant from USDOT to implement a railroad-grade crossing detection system. The system, developed in partnership with CAIT researchers, will be deployed at 50 grade crossings and aboard five light rail vehicles throughout the state. The railroad-grade crossing detection system features multiple cameras on grade crossings and light rail vehicles to record data for an AI computer vision model that monitors and analyzes grade crossing behavior such as near-miss incidents.

For a project recently completed with the Federal Railroad Administration, CAIT researchers examined “stopped-on-track” incidents, which are a leading cause of grade-crossing accidents. During the poster session at the 2024 NJDOT Research Showcase, CAIT’s researchers highlighted a detection system for identifying stopped-on-track incidents and case study examples of how the critical locations can be addressed through design or other interventions. They found that targeted intervention using the AI detection system could reduce stopped-on-track incidents by up to 86 percent.

Visual example of how LiDAR senses the surrounding environment.

Other State DOTs have also started to implement AI-based programs. The Georgia Department of Transportation, in partnership with Georgia Tech, completed a survey of 22,000 road signs around potentially dangerous road curves using AI and vehicle-mounted mobile phone cameras to improve safety at road curves. The Texas Department of Transportation (TxDOT) assessed pavement conditions using LiDAR and AI. TxDOT’s project shares similarities with the research presented by Surya Teja Swarna, but it utilized LiDAR instead of a mobile phone camera.  

In 2022, the Nevada Department of Transportation partnered with the Nevada Highway Patrol, the Regional Transportation Commission of Southern Nevada, and a private technology company to launch an AI-based platform that facilitated the reporting of real-time crash locations. A study on this project found that the AI platform uncovered 20 percent more crashes than previously reported and reduced emergency response time by nine to ten minutes on average while eliminating the need to dial for help.

Recent National Research  

Responses from state DOT officials demonstrate the varied applications of ML solutions. Courtesy of NCHRP.

The National Cooperative Highway Research Program (NCHRP) published a 2024 research report,  Implementing and Leveraging Machine Learning at State Departments of Transportation, that identifies trends in AI transportation research and implementation with a specific focus on machine learning and creates a roadmap for future implementation. The researchers surveyed State DOTs on plans regarding AI, reported case studies of ML implementation by State DOTs, and listed strategies to help DOTs facilitate further inclusion of AI solutions.

The survey of the state DOT officials covered various topics, including the transportation agency’s familiarity with AI methods and tools, types of methods and applications utilized, and challenges in implementation. Among the challenges to implementation, DOT officials noted a lack of public trust, insufficient data collection and storage infrastructure, and, most commonly, scarce labor with knowledge of AI. Most computer and data scientists choose to work in the private sector, and it can be difficult to recruit them to a transportation agency.

The NCHRP report also included multiple case studies from state DOTs such as Nebraska, California, and Iowa, documenting the experiences of these agencies in developing and implementing ML programs.

  • Nebraska DOT (NDOT) used a computer vision Convolutional Neural Network (CNN) algorithm to detect and analyze guardrail quality. NDOT recorded 1.5 million images of guardrail data and used AI to save time and money compared to the manual detection alternative. Among the challenges, NDOT observed that their agency did not have the necessary infrastructure to process large volumes of data and lacked in-house ML expertise. The agency solved the former issue by using a private vendor to process the data and the latter by collaborating with consultants from the University of Nebraska. The algorithm achieved accuracies of 97 percent for guardrail detection and 85 percent for their classification into three types. 
  • The California Department of Transportation (Caltrans) has leveraged AI/ML applications across various projects and partnered with numerous tech companies, including Google. One area of emphasis for Caltrans has been workforce capacity development. While most staff do not have experience with AI-based data analytics, they do have experience with GIS. Caltrans has worked with GIS tool developers to incorporate ML functionalities into the basic user interface of GIS programs, making it more intuitive for their workforce. 
  • Iowa State University, funded by the Iowa Department of Transportation, developed a real-time ML tool to monitor highway performance, enabling a rapid response to traffic congestion. The researchers identified the need for high-performance computing as a significant challenge preventing large-scale implementation. Mass deployment of the tools used in the research study would require a considerable expense, partially due to the stipulation that the code be at least 99 percent reliable. 

For more information on the application and implementation of AI by transportation agencies, the National Academies of Sciences, in collaboration with the NCHRP, published two additional reports in 2024. One, Artificial Intelligence Opportunities for State and Local DOTs: A Research Roadmap, utilizes machine learning methods to analyze research trends in AI and how State DOTs can implement the research. The other, Implementing Machine Learning at State Departments of Transportation: A Guide, serves as a complementary document to the NCHRP report on implementing and leveraging machine learning. 

On a national level, USDOT published its Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence compliance plan in September 2024. USDOT has taken several measures to advance the implementation of AI, including forming an AI Governance Board chaired by the Deputy Secretary and vice-chaired by a new Chief Artificial Intelligence Officer (CAIO), creating an AI Accelerator Roadmap, and providing funds for AI research and implementation.

Lastly, the American Association of State Highway and Transportation Officials (AASHTO) hosted a knowledge session examining the role of AI in transportation in April 2024. Practitioners on the panel highlighted the potential of AI in eliminating the dangerous aspects of data collection and allowing for proactive solutions rather than reactively responding to crashes or injuries.  The panelist discussion touched upon the importance of building trust in a period of rapid AI development, noting the critical role that academic researchers can play as partners with state DOTs to advance and develop the AI technology in ways beneficial for traffic safety and workforce safety, among other topics.

TRID Database 

Artificial Intelligence-based research can be found via TRB’s TRID database. The following are some relevant articles published on recent New Jersey transportation research in AI.

  • Bagheri, M., B. Bartin, and K. Ozbay. (2023). Implementing Artificial Neural Network-Based Gap Acceptance Models in the Simulation Model of a Traffic Circle in SUMO. Transportation Research Record: Journal of the Transportation Research Board, Vol. 2677. https://trid.trb.org/View/2166547 
  • Hasan, A.S., M. Jalayer, S. Das and M. Bin Kabir. (2024). Application of machine learning models and SHAP to examine crashes involving young drivers in New Jersey. International Journal of Transportation Science and Technology, Vol. 14. https://trid.trb.org/View/2162338
  • Hasan, A.S., M. Jalayer, S. Das and M. Bin Kabir. (2023). Severity model of work zone crashes in New Jersey using machine learning models. Journal of Transportation Safety & Security, Vol. 15. https://trid.trb.org/View/2190127
  • Najafi, A., Z. Amir, B. Salman, P. Sanaei, E. Lojano-Quispe, A. Maher, and R. Schaefer. (2024). A Digital Twin Framework for Bridges. ASCE International Conference on Computing in Civil Engineering 2023, American Society of Civil Engineers, pp 433-441. https://trid.trb.org/view/2329319  
  • Nayeem, M., A. Hasan, M. Jalayer. (2023). Investigation of Young Pedestrian Crashes in School Districts of New Jersey Using Machine Learning Models. International Conference on Transportation and Development 2023, American Society of Civil Engineers. https://trid.trb.org/View/2196775  
  • Patel, D., P. Hosseini, and M. Jalayer. (2024). A framework for proactive safety evaluation of intersection using surrogate safety measures and non-compliance behavior. Accident Analysis & Prevention, Vol. 192. https://trid.trb.org/View/2242428
  • Zaman, A., Z. Huang, W. Li, H. Qin, D. Kang, and X. Liu. (2023). Artificial Intelligence-Aided Grade Crossing Safety Violation Detection Methodology and a Case Study in New Jersey. Transportation Research Record: Journal of the Transportation Research Board, Vol. 2677. https://trid.trb.org/VCiew/2169797  
  • Zaman, A., Z. Huang, W. Li, H. Qin, D. Kang, and X. Liu. (2024). Development of Railroad Trespassing Database Using Artificial Intelligence. Rutgers University, New Brunswick, Federal Railroad Administration, 80p. https://trid.trb.org/view/2341095 

Additional Resources

Zone for AI to look for trespassing at railroad crossing

Research Spotlight: Exploring the Use of Artificial Intelligence to Improve Railroad Safety

Posted on

Partnering with the Federal Railroad Administration, New Jersey Transit and New Jersey Department of Transportation (NJDOT), a research team at Rutgers University is using artificial intelligence (AI) techniques to analyze rail crossing safety issues. Utilizing closed-circuit television (CCTV) cameras installed at rail crossings, a team of Rutgers researchers, Asim Zaman, Xiang Liu, Zhipeng Zhang, and Jinxuan Xu, have developed and refined an AI-aided framework for detection of railroad trespassing events to identify the behavior of trespassers and capture video of infractions.  The system uses an object detection algorithm to efficiently observe and process video data into a single dataset.

Rail trespassing is a significant safety concern resulting in injuries and deaths throughout the country, with the number of such incidents increasing over the past decade. Following passage of the 2015 Fixing America’s Surface Transportation (FAST) Act that mandated the installation of cameras along passenger rail lines, transportation agencies have installed CCTV cameras at rail crossings across the country.  Historically, only through recorded injuries and fatalities were railroads and transportation agencies able to identify crossings with trespassing issues. This analysis did not integrate information on near misses or live conditions at the crossing. Cameras could record this data, but reviewing the video would be a laborious task that required a significant resource commitment and could lead to missed trespassing events due to observer fatigue.

Zaman, Liu, Zhang, and Xu saw this problem as an opportunity to put AI techniques to work and make effective use of the available video and automate the observational process in a more systematic way. After utilizing AI for basic video analysis in a prior study, the researchers theorized that they could train an AI and deep learning to analyze the videos from these crossings and identify all trespassing events.

Working with NJDOT and NJ TRANSIT, they gained access to video footage from a crossing in Ramsey, NJ.  Using a deep learning-based detection method named You Only Look Once or YOLO, their AI-framework detected trespassings, differentiated the types of violators, and generated clips to review. The tool identified a trespass only when the signal lights and crossing gates were active and tracked objects that changed from image to image in the defined space of the right-of-way. Figure 1 depicts the key steps in the process for application of AI in the analysis of live video stream or archived surveillance video.

Figure 1. General YOLO-Based Framework for Railroad Trespass Detection illustrates a step-by-step process involving AI algorithm configurations, YOLO-aided detection, and how trespassing detection incidents are saved and recorded to a database for more intensive analysis and characterization (e.g., trespasser type, day, time, weather, etc.)

The researchers applied AI review to 1,632 hours of video and 68 days of monitoring. They discovered 3,004 instances of trespassing, an average of 44 per day and nearly twice an hour. The researchers were able to demonstrate how the captured incidents could be used to formulate a demographic profile of trespassers (Figure 2) and better examine the environmental context leading to trespassing events to inform the selection and design of safety countermeasures (Figure 3).

Figure 2: Similar to patterns found in studies of rail trespassing fatalities, trespassing pedestrians were more likely to be male than female. Source: Zhang et al
Figure 3: Trespassing events were characterized by the gate angle and timing before/after a train pass to isolate context of risky behavior. Source: Zhang et. al

A significant innovation from this research has been the production of the video clip that shows when and how the trespass event occurred; the ability to visually review the precise moment reduces overall data storage and the time needed performing labor-intensive reviews. (Zhang, Zaman, Xu, & Liu, 2022)

With the efficient assembly and analysis of video big data through AI techniques, agencies have an opportunity, as never before, to observe the patterns of trespassing. Extending this AI research method to multiple locations holds promise for perfecting the efficiency and accuracy in application of AI techniques in various lighting, weather and other environmental conditions and, more generally, to building a deeper understanding of the environmental context contributing to trespassing behaviors.

In fact, the success of this AI-aided Railroad Trespassing Tool has led to new opportunities to demonstrate its use. The researchers have already expanded their research to more crossings in New Jersey and into North Carolina and Virginia. (Bruno, 2022) The Federal Railroad Administration has also awarded the research team a $582,859 Consolidated Rail Infrastructure and Safety Improvements Grant to support the technology’s deployment at five at-grade crossings in New Jersey, Connecticut, Massachusetts, and Louisiana. (U.S. DOT, Federal Railroad Administration, 2021) Rutgers University and Amtrak have provided a 42 percent match of the funding.

The program’s expansion in more places may lead to further improvements in the precision and quality of the AI detection data and methods.  The researchers speculate that this technology could integrate with Positive Train Control (PTC) systems and highway Intelligent Transportation Systems (ITS). (Zhang, Zaman, Xu, & Liu, 2022) This merging of technologies could revolutionize railroad safety. To read more about this study and methodology, see this April 2022 Accident Analysis & Prevention article.

References

Bruno, G. (2022, June 22). Rutgers Researchers Create Artificial Intelligence-Aided Railroad Trespassing Detection Tool. Retrieved from https://www.rutgers.edu/news/rutgers-researchers-create-artificial-intelligence-aided-railroad-trespassing-detection-tool

NJDOT Technology Transfer. (2021, November 8). How Automated Video Analytics Can Make NJ’s Transportation Network Safer and More Efficient. Retrieved from https://www.njdottechtransfer.net/2021/11/08/automated-video-analytics/

Tran, A. (n.d.). Artificial Intelligence-Aided Railroad Trespassing Data Analytics: Artificial Intelligence-Aided Railroad Trespassing Data Analytics:.

United States Department of Transportation: Federal Railroad Administration. (2021). Consolidated Rail Infrastructure and Safety Improvements (CRISI) Program: FY2021 Selections. Retrieved from https://railroads.dot.gov/elibrary/consolidated-rail-infrastructure-and-safety-improvements-crisi-program-fy2021-selections

Zaman, A., Ren, B., & Liu, X. (2019). Artificial Intelligence-Aided Automated Detection of Railroad Trespassing. Journal of the Transportation Research Board, 25-37.

Zhang, Z., Zaman, A., Xu, J., & Liu, X. (2022). Artificial intelligence-aided railroad trespassing detection and data analytics: Methodology and a case study. Accident Analysis & Prevention.