Associate authorization to the data models

We are trying to build an enterprise level data model layer with multiple flavors of data sources below. This is not just for developers or applications. Our idea is that all access to the underneath data needs to go through the data models, so we would like to associate data access authorization to the models, so when a user or application access the data through a specific model, authorization associated to that model will be enforced. How can that be done? Thanks.