Numerous Classic and emerging applications uses graph processing as core component. NVIDIA proposed CUDA (Computed unified device architecture) which increases the computation substantially with power of GPU. Graph algorithm is simplest way to improve the graph processing performance. In spite of improve programmability of GPU, writing efficient and correct program is very difficult and the task become more challenging and complex in case of graph due to irregular structures and large size of graph data. We proposed few fundamental graph algorithms like BFS, shortest path algorithm which efficiently accelerate and optimized graph processing with GPGPU (general Purpose graphical processing unit) technique.