The aim of this ongoing research project is to reach a pattern recognition process (design system) that can turn architecture to music, and turn music to architecture. In both operations, style and character will be an essential issue in the design of this process system. Based on the artistic relationship between architecture and music, we shall try to settle a mechanism for a pattern recognition audio visual system that can that can turn a certain style of architecture to a collection of melodies that truly reflects the character of that architecture. This system can also perform the inverted process that is turning a certain musical style to a collection of shapes and forms. In this regard it is important to point out to the inter relationship between the architectural design process and writing a musical note. A musician composing a melody works from above, that is from the whole to the parts. A composer does not usually put notes together in order to get some melody he envisages the character of a melody in a statu nascendi and proceeds from above as he tries to concretize it in all its part. In this respect, the articulation of the whole and the important junctures of the parts, the beginning(s), or the end(s), the composer accepted the attitude of his contemporaries, and could therefore integrate principles of architecture such as the golden section and the Pythagorean Theorem, along with principles of acoustic harmony. The melody steps of these musical golden sections (time space intervals of varying scales) can follow the rules of the Pythagorean Theorem. Originality in expression does not depend on invention of new words nor originality in poetry on invention of new measures nor in painting on invention of new colours, or new modes of using them. That means that innovation and creativity is a matter of "relationships". In architecture relationships exist in two ways, in the environment itself, and in the individual's ability to understand and relate to them. They exist at a real, concrete level where the individual is aware of them through his senses, perception, hearing, touching, etc. And the also exist at an abstract or conceptual level in the actual object. They cannot be seen or heard even though the can be described. At the end we are referring to "linguistics" as a platform that can consist architecture and music i.e. a perceptual or surface structure and a conceptual or deep structure that combines both architecture and music. Architecture and music share the unification of three realms, conceptual, objectual and perceptual. In the historical writings of ikhwan al safa, (The Brethren of Sincerity Muslim philosophers in Basra, Iraq, in the 10th century), the relationship between arts, architecture, music, astronomy, mathematics and chemistry was clearly mentioned. In this paper we had chosen Andalusian Architecture as a part of Islamic Architecture that existed in Spain (AD 711 1492). Also there is a significant retrieving and re composing for Andalusian music prepared by Al Rahbaniya in Lebanon in seventies of the 20th century, also Andalusian music is still played in Morocco, Algeria and Tunisia.