Why IoT development methodology is an open problem
Internet of Things (IoT) systems combine sensors, actuators, embedded processors, networks, cloud or edge services and, increasingly, data-driven decision-making. Early surveys catalogued the enabling technologies and protocols (Al-Fuqaha et al., 2015), while software engineering researchers argued that IoT needs its own abstractions and development practices rather than borrowed ones (Zambonelli, 2017; Larrucea et al., 2017).
Hornos and Quinde (2024) take up that argument. Their paper, published in the Journal of Reliable Intelligent Environments, asks whether existing development methodologies, many inherited from conventional information systems, fit the heterogeneous and evolving nature of IoT systems. They do not propose a new methodology. Instead they review existing ones, identify recurring challenges and set out research directions. For a doctoral researcher in software engineering or IoT, the paper is a useful map, but its conclusions need to be read in light of how its evidence was gathered.
What the paper reports
A four-stage life cycle as the yardstick
The authors first review general approaches, including the Waterfall and Spiral models and agile methods such as Scrum combined with Extreme Programming, and life-cycle standards such as ISO/IEC/IEEE 12207 (software) and ISO/IEC/IEEE 15288 (systems) (Hornos & Quinde, 2024). From these sources they derive a four-stage life cycle that they use to compare IoT methodologies.
| Stage | Activities in Hornos and Quinde's framework | Coverage reported in the reviewed methodologies |
|---|---|---|
| Conception | Planning; requirements elicitation | Addressed unevenly; non-functional requirements are a recurring weakness |
| Modelling | Analysis; design; model verification; simulation | Design is the most frequently mentioned activity |
| Construction | Coding or implementation; integration; testing; deployment | Coding and implementation are commonly covered |
| Post-construction | Management or operation; maintenance; decommissioning | System management appears in only a couple of works; decommissioning in only one methodology |
Four families of IoT methodologies
IoT-specific proposals are grouped into four families: methodologies based on (meta)modelling, such as MDE4IoT and ROOD; service-oriented methodologies, such as SERVUS; agent-oriented methodologies, such as ELDAMeth; and methodologies based on other approaches, including the test-driven TDDM4IoTS and an interaction-flow approach for IoT user interfaces (Hornos & Quinde, 2024). The central finding is that no single methodology comprehensively covers the whole life cycle, and that post-construction activities are the most neglected.
Challenges and research directions
The paper identifies five challenge areas: heterogeneity and interoperability, covering both hardware and software integration; scalability, adaptability and integration with emerging technologies; security and privacy; user-centric approaches; and reliability or dependability. It then proposes five research avenues: more integrative and holistic methodologies, support tools that help developers apply them, progress towards widely used standard methodologies, integration of artificial intelligence and machine learning, and more intuitive, user-centred tools (Hornos & Quinde, 2024). A notable observation is that most development methodologies are used only by the people who created them.
Strengths
- A clear comparative frame. Anchoring the comparison in a life cycle derived from international standards makes gaps visible and repeatable: another researcher can apply the same four stages to a new methodology.
- A substantive finding about the life cycle. The neglect of operation, maintenance and decommissioning is not a minor omission. IoT devices remain in the field for years, and their software must be updated and eventually retired safely.
- Challenges framed as engineering problems. Heterogeneity, non-functional requirements and dependability are presented as concrete obstacles for methodologies, not as general concerns about IoT.
- Honest about adoption. Noting that most methodologies are used only by their creators points to a real weakness of the field and invites empirical work.
Limitations and methodological issues
How the literature was found
The paper is presented as a literature review. As far as the published text shows, it does not report a reproducible search protocol: the databases searched, search strings, date range, or inclusion and exclusion criteria. Software engineering has established guidance for systematic mapping studies (Petersen et al., 2015) and for snowballing from reference lists (Wohlin, 2014). Without such reporting, readers cannot judge whether the set of methodologies is complete or whether some families are over-represented.
Feature comparison is not evidence of effectiveness
Methodologies are compared on what they claim to support, not on how they perform. A methodology may describe a maintenance phase without that phase working well in practice, and a narrowly scoped methodology may succeed in its own domain despite covering fewer stages. Case study research, with explicit propositions, multiple data sources and stated threats to validity (Runeson & Höst, 2009), is the natural next step for testing the paper's claims.
Practitioner knowledge is largely outside the frame
Much IoT development practice is documented in industry reports, platform documentation and practitioner blogs rather than journals. Multivocal literature reviews, which include such grey literature with explicit quality criteria (Garousi et al., 2019), could test whether the academic picture of IoT methodology matches what development teams actually use.
Proximity to the authors' own work
One of the reviewed methodologies, TDDM4IoTS, was developed by a team that includes Hornos (Guerrero-Ulloa et al., 2020). Its later validation used a case study of an assistive system for visually impaired people and a survey of developers, most of them engineering students (Guerrero-Ulloa et al., 2023). This is normal in a small field, but a review that includes its authors' own methodology should say so and describe how bias was managed. It also illustrates the paper's own point: evaluations are often run by the methodology's creators.
Topics outside the five challenge areas
Environmental sustainability, such as device energy use and electronic waste at decommissioning, is not among the paper's challenge areas, although it follows naturally from its finding about post-construction stages. Organisational factors, such as team skills, budgets and procurement, receive less attention than technical ones, apart from the paper's remarks on interdisciplinary teams.
How the paper sits in the wider literature in 2026
The paper extends earlier reviews rather than replacing them. Fortino et al. (2021) reviewed methodologies, frameworks, platforms and tools for IoT viewed as a system of systems, and also found fragmentation. Hornos and Quinde add a sharper life-cycle lens and an explicit research agenda.
Two developments since the paper was written strengthen its argument about post-construction stages. First, the second edition of the IoT reference architecture standard, ISO/IEC 30141:2024, provides a common vocabulary and architectural viewpoints that a methodology can map to (ISO/IEC, 2024). Second, the EU Cyber Resilience Act, which entered into force on 10 December 2024, requires manufacturers of products with digital elements to handle vulnerabilities and provide security updates over a support period. Its reporting obligations apply from 11 September 2026 and its main obligations from 11 December 2027 (European Parliament & Council of the European Union, 2024). For IoT products on the EU market, maintenance is therefore becoming a regulated activity, not an optional phase.
Worked example: turning a research direction into a doctoral study
The following example is illustrative and hypothetical. A doctoral researcher wants to address the neglected post-construction stage for an indoor air-quality monitoring system in university buildings.
- Narrow the claim. Instead of "a complete IoT methodology", the aim becomes "extending an existing methodology with explicit operation, update and decommissioning activities".
- Map the baseline. Apply Hornos and Quinde's four stages to two candidate methodologies and record, activity by activity, what each specifies.
- Define outcomes in advance. For example: time to deploy a security update across all devices, number of unplanned site visits per year, and completeness of the decommissioning record.
- Choose a design. A multiple case study with two development teams, following case study reporting guidance (Runeson & Höst, 2009), with interviews, project artefacts and system logs as data sources.
- Link to current requirements. Map the extended activities to the support-period and vulnerability-handling duties in the Cyber Resilience Act (European Parliament & Council of the European Union, 2024) and to the viewpoints of ISO/IEC 30141 (ISO/IEC, 2024).
- Report threats to validity. State that the researcher designed the extension, and describe how independent teams and pre-specified metrics reduce that bias.
Whatever the scenario, the same reporting questions apply. Table 2 summarises what a methodology evaluation should make explicit, drawing on case study guidance (Runeson & Höst, 2009) and the gaps identified by Hornos and Quinde.
| Item to report | Why it matters | Example of adequate reporting |
|---|---|---|
| Life-cycle stages covered | Shows which claims the study can and cannot support | Mapping of each methodology activity to conception, modelling, construction and post-construction |
| Setting and participants | Results from student projects may not transfer to industry | Team size, experience, domain and whether participants were students or practitioners |
| Hardware and platforms | Heterogeneity is the main IoT challenge | Device types, communication protocols, edge or cloud services used |
| Non-functional requirements | Security, reliability and energy use are easily lost in user stories | How each requirement was specified, verified and measured |
| Outcome measures | Prevents choosing favourable metrics after the fact | Measures defined before data collection, with how and when they were recorded |
| Researcher role | Creator involvement can bias evaluation | Statement of who designed the methodology and who ran the evaluation |
Common mistakes when building on this paper
- Citing the paper as evidence that a particular methodology "works" or "fails"; it compares coverage, not outcomes.
- Describing its categories incorrectly; the IoT families are (meta)modelling-based, service-oriented, agent-oriented and other approaches.
- Proposing a methodology that covers every stage and every challenge at once, which cannot be evaluated within a doctorate.
- Treating a survey of student developers as proof of industrial effectiveness.
- Ignoring standards and regulation that now shape IoT maintenance and security.
What this means for doctoral researchers
Hornos and Quinde provide a reliable vocabulary and a credible list of gaps. The most defensible doctoral contributions will take one gap, most promisingly post-construction support, interoperability or the handling of non-functional requirements, and test a methodological change in real projects with pre-defined measures. A literature chapter that builds on this paper should report its own search protocol, include practitioner sources where relevant, and date every standards or regulatory claim.
It is also worth using the paper as a model of structure rather than only as a source. Its sequence of background, comparative framework, challenges and research directions is a sound outline for a methodological literature review chapter. What a thesis chapter should add is the evidence trail the paper leaves implicit: how sources were found, why each methodology was included, and how confident the reader can be in each comparison. Where a judgement rests on a single proposal paper written by the methodology's own authors, say so; where independent evaluations exist, give them more weight.
Finally, keep the system and its context in view. An IoT methodology for a hospital, a farm and a factory will face different dependability, privacy and maintenance demands. Hornos and Quinde's five challenge areas are a useful checklist for scoping, but the doctoral contribution comes from showing how a methodology handles those challenges in one well-described setting.
Key takeaways
- Hornos and Quinde (2024) find that no reviewed methodology covers the whole IoT life cycle, with post-construction activities the least supported (Hornos & Quinde, 2024).
- Their comparison is based on what methodologies describe, so it identifies gaps but does not show which methodology performs best.
- The review does not report a reproducible search protocol, which limits confidence in its coverage (Petersen et al., 2015).
- The EU Cyber Resilience Act makes vulnerability handling and security updates part of the regulated life cycle for many IoT products (European Parliament & Council of the European Union, 2024).
- A strong doctoral project narrows one research direction and evaluates it empirically with pre-specified measures (Runeson & Höst, 2009).
