When a reference or a pointer is passed as an argument into a function, a thing that is the size of a pointer must be passed. (References are, under the covers, pointers. They just have a different syntax for using them.) When you call by value, you’re passing a copy of the data.
In a 32-bit environment, a pointer is usually four bytes wide. In a 64-bit environment, a pointer is usually eight bytes wide. In a 16-bit environment, a pointer is usually two bytes wide. I say usually, because the size of a pointer and of most other data types can vary across compilers and versions, so never make assumptions about the size of a pointer or anything else. So, the cost of passing a reference or pointer is the cost of passing the number of bytes that a pointer occupies in your environment. And then, of course, once you’re inside the called function, accessing the data will require one level of indirection through the address.
Now, let’s contrast this with the cost of passing an argument by value. When you pass by value, you are passing a copy of the data. So, if an int is four bytes (but that various from one compiler to another, so check first), then the cost of passing an int by value is four bytes. If a double is eight bytes, the cost of passing a double is eight bytes. The cost of passing a structure or class by value that occupies 128 bytes will be the cost of copying and passing 128 bytes. (If what you’re passing has a copy constructor, you also incur the cost of running that. If there’s a destructor, that cost will eventually be incurred as well.)
On the surface, it looks like the cost of passing something by value whose size is less than a pointer will cost less than passing by reference/pointer, while the cost of passing something by value whose size is greater than the size of a pointer will be higher. But remember that once you’re inside the called function, the copy of the data might be accessed directly, without indirection using an address. And there might be copy constructor and/or destructor action to account for as well.
Your question asks which one is faster. Raw speed is affected by many factors, including CPU clock speed, cache organization and current state, RAM speed, etc. And calling conventions also vary from one environment to another. Some calling conventions place all arguments on the stack, while others pass the first few arguments in general-purpose registers. What can actually be passed in those registers by value depends on the sizes of those registers. And the pattern of accessing the data inside the called function will also play a role in overall speed.
So, if you’re really interested in the relative speed of passing by reference/pointer and passing by value for some specific, you need to profile the code, keeping all the other speed-altering variables unchanged.
As a rule of thumb, passing by reference or pointer is typically faster than passing by value, if the amount of data passed by value is larger than the size of a pointer. But there are many factors to consider, as you can see in the discussion above.
Of course, if your called function needs to modify the data, your decision is already made for you…you need to pass by reference or pointer. Modifying a copy of the data (passed by value) isn’t going to do anything useful for you.