On Youtube

On this image the base(2) and the barrel(3) is shown whit their intended rotations around their axis. The base rotates on the y axis and the barrel moves up and down on the x axis.
It is important to have the z axis point forward, since that is what Unity assumes to be forward. (I discovered that with trail and error instead of just reading up on the subject)

Hierarchy

When that was done i needed to place it in a proper hierarchy. The Top is an empty GameObject. In there I put an other empty GameObject and named it Sides, and added 2 pillar cubes to it. The barrel is a cube renamed to CenterPiece, the cube inside it is to show where to front is.

Scripting

We have two rotating parts. The rotation is done independently from each other but together they will point at an object.

Got values?

I got the rotation values towards the other object with the neat little Unity function: LookAt. Lookat points the z axis towards the object.
I only need two rotation values. So I saved the values before the lookAt, called lookAt, saved the new rotation values, and set the rotation back to the old values before rotating.

Turning

I made two separate scripts for the base and barrel, while the code is very similar there are slight differences like the axis.
I use a simple transform.Rotate with the help of Mathf.MoveTowardsAngle to smootly turn to only on the required angle. I first tried turning with slerp, but the barrel and base rotated slightly out of sync making the whole thing look terrible. MoveTowardsAngle also gives a nice constant rotation speed, which was exactly what I was looking for.