Yes, it can. If dirt, mud, leaves, or mulch are allowed to accumulate and decompose on the surface, they can settle into the open-cell pores.
This block of debris will clog the interconnected void spaces, reducing the water drainage rate of your pervious concrete pavement.