[postgis-devel] Proposal for sensible number output semantics

[rmrodriguez at carto.com]

Ok, I see that the proposal is basically the same but I have a different
way to describe it.

I got confused partly because I don't make the distinction at 15 precision
and I always return the sortest with up to N decimal digits of precision,
so 15 is not special but 17 will be the max digits in any case.

By modifying ryu, I have implemented a method that returns the shortest
decimal representation for our use case (<1e15) although it needs some
optimization (adapting the ones already in ryu to print the integer parts).

The thing is currently missing is the "truncate precision"  as the one
present in ryu is for all digits, so I might need to add a second one to
round only the decimal digits to the desired precision.


If nothing unexpectes happens I hope I'll have a fulk implementation soon
so we can review the output diffences with the 3.0 output and confirms
that's the behaviour thst we want. I would rather not write a RFC until we
have something we are happy with that can be applied to all text output
functions.



On Saturday, April 18, 2020, Martin Davis <mtnclimb at gmail.com> wrote:

>
>
> On Sat, Apr 18, 2020 at 12:22 PM <rmrodriguez at carto.com> wrote:
>
>> On Sat, Apr 18, 2020 at 6:18 PM Martin Davis <mtnclimb at gmail.com> wrote:
>>
>> > When you say "limit" do you mean truncate or round?   I don't think
>> truncation is a good approach (based on what every other precision-limiting
>> system I have seen does).
>> > If you mean round then I *think* your suggestion is essentially the
>> same as mine.
>>
>> Ideally, rounding.
>>
>
> Ah, that's good.
>
>>
>> > I would say precision 0..8 is most useful; beyond that not as much.
>> >
>> > But surely there has to be some transition point at which rounding
>> stops and "full" precision is used?
>>
>> Yes, but rounding might stop at different points
>>
>
> My idea is that 15 decimal digits is about the limit of DP precision.
> Below that rounding would be noticeable, beyond it it would not be.  It
> could be 14 or 13 instead, I think.  The key point is that the current
> default will produce full precision.
>
> >> - There is no artificial distinction between < 15 and > 15, which
>> >> eliminates inconsistencies. Note that in your proposal you would get
>> >> odd things like this:
>> >>   - 0.300000011920928955078125 with precision 10 -> 0.3000000119
>> >>   - 0.300000011920928955078125 with precision 20 -> 0.3 (since 0.3 has
>> >> the same double representation)
>> >
>> >
>> > I don't see how  0.300000011920928955078125 has the same DP
>> representation as 0.3. DP has about 16 digits of precision, so surely the
>> first number would be represented as (close to) 0.300000011920929 ?
>>
>> Sorry, I took a 32bit float example (from tryu) so it doesn't work
>> with 64bits. Let's see a 64 bit double example:
>> ```
>>
>> postgis=# Select ST_AsText(ST_MakePoint(0,
>> 0.299999999999999988897769753748), 30);
>> st_astext
>>
>> POINT(0 0.299999999999999988897769753748)
>> (1 row)
>>
>> postgis=# Select ST_AsText(ST_MakePoint(0, 0.3), 30);
>> st_astext
>>
>> POINT(0 0.299999999999999988897769753748)
>> (1 row)
>> ```
>>
>
> Okay, that makes more sense.
>
>
>>
>> In both those examples, for anything except precision 0, IMO the
>> output should be POINT(0 0.3). Why? Because both
>> 0.29999999999999998889776975374 and 0.3 have the same double
>> representation (in hex 0x3fd3333333333333) so when you import 0.3 you
>> are getting the same value, thus it makes sense to output the shortest
>> representation in all cases.
>>
>
> Yes, and I think that will happen with the proposed scheme?
>
>>
>> From what I understood of your proposal, there would be a precision
>> (17+) where you would get more digits beyond 0.3. In my proposal you
>> would never get any more digits.
>>
>
> No, at 15 and above requested digits you would get "full precision" and no
> more.
>
> The intent is NOT to allow the user to force a meaningless precision to be
> output.  There are two goals:
>
> a) allow output of numbers with a textual precision that matches the known
> DP precision.  For instance, if ST_SnapToGrid(geom, .01) is used then
> output should only show 2 decimal digits (even if roundoff actually causes
> a bit of drift in the internal representation.
> b) allow output of full (round-trippable) precision, with that as the
> default
>
> Part of the motivation for (a) is that I hoping that "soon" PostGIS will
> gain the ability to execute some operations to a specified precision (in
> particular overlay, and also a better geometry rounding capability) .  This
> will make moving data in out of PostGIS more robust, since it will prevent
> creating geometry in PostGIS that carries excess precision and then becomes
> invalid when exported at a lower precision.
>
> Ideally we should capture all this as an RFC, and then (if implemented)
> document it in the manual (with examples).
>


-- 
Raúl Marín Rodríguez
carto.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20200419/ee3a9e98/attachment-0001.html>