Building structural works require regular inspections and monitoring of damage progression to ensure compliance with relevant standards and jurisdictional requirements. Conventional quality inspections rely on manual measurements, which is costly, tedious, and error-prone. Recently, Terrestrial Laser Scanners (TLSs) have shown promising performance in terms of accuracy, cost, and efficiency for inspecting building structural members. Nevertheless, utilizing TLS for quality inspection of building structural members lacks generalizable and efficient approaches to inspect and monitor various quality criteria for different types of building structural members over time. To fill this gap, a generalizable framework for inspecting and monitoring building structural members using multi-temporal point clouds is developed in this study. First, an informative cross-section shape and structural member type invariant representative plane from each building structural member, which preserves the underlying dimensional imperfection-related features, is extracted. Then, a combination of geometric imperfections in building structural members, including deflection and slope in beams and inclination and straightness in columns adopting the standard practice and definitions in building codes and standards, are identified and quantified. Finally, a change detection method is proposed to monitor the geometric quality of building structural members over time. Experiments on real-world multi-temporal point clouds of a building under renovation are performed to validate the performance of the proposed framework by comparing the calculated deformations with the field measurements. An average Mean Absolute Error (MAE) of 1.59 mm ± 0.72 mm and an average MAE of 0.67 mm ± 0.25 mm were reported for building structural member deflection and slope deviation, respectively, compared to manual measurements. The basis of the presented methodology framework, including representative plane detection and compliance checks criteria, can be extended for various geometric quality compliance checks for different building structural members.