This paper proposes a vision chip hybrid architecture with dynamically reconfigurable processing element (PE) array processor and self-organizing map (SOM) neural network. It integrates a high speed CMOS image sensor, three von Neumann-type processors, and a non-von Neumann-type bio-inspired SOM neural network. The processors consist of a pixel-parallel PE array processor with O(N×N) parallelism, a row-parallel row-processor (RP) array processor with O(N) parallelism and a thread-parallel dual-core microprocessor unit (MPU) with O(2) parallelism. They execute low-, mid- and high-level image processing, respectively. The SOM network speeds up high-level processing in pattern recognition tasks by O(N/4×N/4), which improves the chip performance remarkably. The SOM network can be dynamically reconfigured from the PE array to largely save chip area. A prototype chip with a 256 × 256 image sensor, a reconfigurable 64 × 64 PE array processor/16 × 16 SOM network, a 64 × 1 RP array processor and a dual-core 32-bit MPU was implemented in a 0.18 μm CMOS image sensor process. The chip can perform image capture and various-level image processing at a high speed and in flexible fashion. Various complicated applications including M-S functional solution, horizon estimation, hand gesture recognition, face recognition are demonstrated at high speed from several hundreds to >1000 fps.