I mean, if we are all conspicuous and identified where is the need for controlled airspace?
You'll still need it in busier areas - yes, busier than the visual circuit or instrument procedure will allow. Otherwise it'll be complete chaos if we all tried to all negotiate with each other to land onto a runway. Factor in different aircraft performance, mixed IFR and VFR operations, wake turbulence, changing weather, changing runway, someone shouts for help, etc.
You get the picture..
Someone's still going to need to sort us all out and keep the airspace and runway usage optimised.
If we don't take part in protecting GA access to infrastructure, the smaller aerodromes will be built over, the larger aerodromes will have exorbitant handling fees and awkward procedures, and airspace will be over-classified to Class A everywhere.