As a key node in international trade, the operational efficiency of container terminals directly affects the turnaround time of ships and logistics costs. Bay allocation and yard crane scheduling are two crucial aspects in container terminal operations, which are interdependent and influence each other. Therefore, this paper proposes a method based on the deep reinforcement learning Actor-Critic framework to study the collaborative optimization of bay allocation and yard crane scheduling, aiming to achieve collaborative decision-making in these two areas. On the one hand, a mixed-integer programming model is established considering factors such as yard crane operation balance, non-operational time of yard cranes, operation time of yard cranes, waiting time of internal container trucks, and container reshuffling volume. On the other hand, by constructing the state space of the container yard, the action space of yard cranes, and a multi-objective reward function model, a solution algorithm based on the deep reinforcement learning Actor-Critic framework is proposed. Through comparative analysis of different-scale examples, it is verified that the proposed algorithm outperforms the traditional genetic algorithm in terms of solution quality. Compared with existing research results, under the condition of maintaining the operation balance of yard cranes, the new algorithm can effectively reduce the operation and non-operational time of yard cranes, shorten the waiting time of automated guided vehicles, and lower the container reshuffling rate.