Abstract: Well maintained cycleways will encourage more people to cycle, as the condition of cycleways is important for the safety, accessibility and riding comfort of cyclists. Despite that, only a few models used to describe the quality of service for cyclists take the surface condition into account. Objective measuring methods are needed to enable reliable and effective assessment of surface conditions, and measurable performance criteria related to the needs of cyclists should be developed. The purpose of this study has been to test the reliability and validity of using accelerometers in smartphones to assess the riding comfort on cycleways. A smartphone application converting three-dimensional accelerometer measurements into a single indicator for cycleways has been used to assess road surfaces in two field studies, in Sweden and Norway, respectively. Both studies assessed test sections of varying quality. To relate the measurements to subjective riding comfort assessments by cyclists, recruited cyclists collected quantitative data using the app, whilst also rating their perceived riding comfort by completing a survey. Measurements were also related to standard road surface condition indicators, generated from a road surface tester equipped with 19 laser sensors: international roughness index (IRI), mega- and macrotexture. The results show that it is possible to describe the unevenness of a cycleway using the technology present in smartphones. A software application can be used to collect and analyse data from the acceleration sensors in the phone, which can then be used to describe the riding comfort of cyclists. It is mainly the unevenness in the 50–1000 mm size-range that create the greatest discomfort for cyclists, and intermittent vibrations are perceived as more uncomfortable than more evenly distributed vibrations. Therefore, IRI is not a relevant measurement for describing the riding comfort of cyclists.