I am new to GIS and am testing a few workflows and needed some advice on them. We are dealing with about 30 million rows of user location data per day (lat, lon) in a Parquet file that can be converted to CSVs. We also have the counties/state shapefile for the US.
I initially used ArcGIS to intersect the two and tag the lat lon to the state using the interface on ArcGIS. We realized it might not be a great idea to user the interface on large data and decided to move towards an entirely Python workflow.
Can anyone tell me how to tag a state to a lat/lon similar to the intersect command on ArcGIS(need attributes from both tables) using open source geospatial libraries in Python? Another condition to be aware of is that we want to assume that a point can fall in two states (we are going to expand the same workflow for other use cases like checking if a person visited a store etc)
أكثر...
I initially used ArcGIS to intersect the two and tag the lat lon to the state using the interface on ArcGIS. We realized it might not be a great idea to user the interface on large data and decided to move towards an entirely Python workflow.
Can anyone tell me how to tag a state to a lat/lon similar to the intersect command on ArcGIS(need attributes from both tables) using open source geospatial libraries in Python? Another condition to be aware of is that we want to assume that a point can fall in two states (we are going to expand the same workflow for other use cases like checking if a person visited a store etc)
أكثر...