Modeling surface water quality using soft computing techniques is essential for the effective management of scarce water resources and environmental protection. The development of accurate predictive models with significant input parameters and inconsistent datasets is still a challenge. Therefore, further research is needed to improve the performance of the predictive models. This study presents a methodology for dataset pre-processing and input optimization for reducing the modeling complexity. The objective of this study was achieved by employing a two-sided detection approach for outlier removal and an exhaustive search method for selecting essential modeling inputs. Thereafter, the adaptive neuro-fuzzy inference system (ANFIS) was applied for modeling electrical conductivity (EC) and total dissolved solids (TDS) in the upper Indus River. A larger dataset of a 30-year historical period, measured monthly, was utilized in the modeling process. The prediction capacity of the developed models was estimated by statistical assessment indicators. Moreover, the 10-fold cross-validation method was carried out to address the modeling overfitting issue. The results of the input optimization indicate that Ca2+, Na+, and Cl− are the most relevant inputs to be used for EC. Meanwhile, Mg2+, HCO3−, and SO42− were selected to model TDS levels. The optimum ANFIS models for the EC and TDS data showed R values of 0.91 and 0.92, and the root mean squared error (RMSE) results of 30.6 µS/cm and 16.7 ppm, respectively. The optimum ANFIS structure comprises a hybrid training algorithm with 27 fuzzy rules of triangular fuzzy membership functions for EC and a Gaussian curve for TDS modeling, respectively. Evidently, the outcome of the present study reveals that the ANFIS modeling, aided with data pre-processing and input optimization, is a suitable technique for simulating the quality of surface water. It could be an effective approach in minimizing modeling complexity and elaborating proper management and mitigation measures.