Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 13/02/20 9:17 AM, Michael Torrie wrote:
On 2/12/20 7:44 AM, Ethan Furman wrote:
On 02/11/2020 04:38 PM, Michael Torrie wrote:
...
True. Costs can be calculated and planned for. But Technical debt is
often impossible to quantify in a real, meaningful, business sense,
other than the that we "know" it's going to cost big time in the future.
In some senses, it's theoretical future cost. That's what I was having
a hard time with, and still do to a large degree.
I think an oft overlooked aspect of technical debt is the affect on
the programmers dealing with it: frustration, burn-out, and
job-seeking.
Yeah for sure. A programmer may not love dealing with technical debt,
and certainly he may want to chase greener fields. But there are lots
of things about lots of jobs that are less pleasant than other things.
Personally I tend to get way caught up trying to get a perfect design
that avoids technical debt, but often get not much done (on my personal
projects). Whereas another guy just cranks out code that isn't always
the best but serves his needs at the time. Guess who is more productive
overall? Not me.
Applauding your professional attitude, but haven't you (only) defined
"productive" as purely getting 'this job' out the door? cf any
definition of "Total" cost?
I had an IT Manager moaning at me precisely because of one of those
'other guys'. The job had been 'done'. The contract had mere 'words' to
cover 'quality'. Accordingly, difficult to argue that 'the guy' hadn't
done what was asked (my 'specialist' $advice on the matter). However,
the code was 'ramshackle' and had obviously been 'dashed off' in the
shortest amount of time possible. Maintenance costs were expected to be
high; motivation of in-house staff to do so, expected to be v.low!
So, we're re-negotiating the contract-wording... More importantly,
improving their acceptance testing, and adding more detail to their
specs/requirements, etc. (the mistake there, was thinking that reqmts to
external staff would be 'the same' as such to in-house employees)
However, the interesting question was to ask whether he (the manager)
was 'taking advantage' of the "gig economy". (yes, that's what it's
there for!) Thus, what relationship did he (or even, 'they' of the IT
dept) have with the contractor? The home truth: 'using' the cheapest
people creates a 'race to the bottom', and must, by definition, lead to
'the cheapest job' being done. Why the surprise?
(He doesn't hire me as a programmer - and we're not even talking
'Python', apologies!)
I asked him if he wanted a 'professional' to do the job, ie in a
'professional manner'. Yes! So, why are you looking for the guy(?)
who'll do it at the lowest-possible price/rate or imposing the delivery
date on a short time-line? Ahh...
The above concerns contractors, but I managed to slide-in a few
questions/comments that relationships with in-house staff require
similar consideration and attention!
OK, so turning it around to people more like 'us': Professionalism? As a
programmer/coder, would you rather work on decent code? So, is that what
you turn-out? Alternatively, vice-versa.
That is a slightly unfair, or biased, question. I am in the fortunate
position of being able to tell certain people that I won't undertake
their work, that where they want me to work will push the price UP, that
their tool-set is sub-standard, or that the existing platform is imbued
with uncomfortably high 'risk'. I quite understand that someone who
'needs the job' will see that as a 'luxury'. (apologies as necessary)
That said, would you build a team with a mixture of 'clock-watchers' and
'professionals' or would/do you enjoy working within such a team?
Both 'sides' need to understand that there are two view-points to every
relationship.
One thought that occurred to me lurking/reading this thread, is the
likelihood that many of the differing views arise between those working
in an 'Agile' environment, and those not (or a pseudo-/'we call
it'-Agile environment)?
One of the (social) contracts in Scrum (for example) is that the
professionals estimate the time needed to do the job, cf some (likely
ignorant) 'manager' imposing a deadline. In my experience (YMMV) an
established, professional 'Agile' team tends to follow practices which
attempt to avoid adding to Technical Debt, and factor-in any need to
correct sub-standard or sans-tests old-code. There is no doubt in my
mind, that the participation of 'users' helps to create an understanding
that (like anything else) software takes time (read also $), and that
over-loading (?abusing) development staff is itself a 'Technical Debt'
("Marginal Cost") for the organisation! (absences, turn-over, ... you've
been-there, seen-that...). There again, would user-members of the team
notice if 'we'd' padded our estimates so as to have plenty of time for
'other things'?
It's by no means a cut-and-dried situation. What 'works
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/12/20 7:44 AM, Ethan Furman wrote: > On 02/11/2020 04:38 PM, Michael Torrie wrote: > >> It's all just different ways of accounting for the same things. In >> the olden days before the term "technical debt" was invented, we >> called this "total cost of ownership." > > TCO is not a fixed number. For example, if a loan is taken to help > fund a project, then the "interest debt" will be a portion of the > TCO, but its amount will vary depending on the interest rate: 15% > will be more interest debt than 4%. Likewise, the technical debt for > a project will be higher or lower depending on the quality of the > code written. True. Costs can be calculated and planned for. But Technical debt is often impossible to quantify in a real, meaningful, business sense, other than the that we "know" it's going to cost big time in the future. In some senses, it's theoretical future cost. That's what I was having a hard time with, and still do to a large degree. > I think an oft overlooked aspect of technical debt is the affect on > the programmers dealing with it: frustration, burn-out, and > job-seeking. Yeah for sure. A programmer may not love dealing with technical debt, and certainly he may want to chase greener fields. But there are lots of things about lots of jobs that are less pleasant than other things. Personally I tend to get way caught up trying to get a perfect design that avoids technical debt, but often get not much done (on my personal projects). Whereas another guy just cranks out code that isn't always the best but serves his needs at the time. Guess who is more productive overall? Not me. More on topic, I feel like the technical debt issue might be over used to push folks to upgrade from Python 2, or to do promote certain technologies. Without knowing the details of how and why they used it, it's impossible for me to say that it will cost them a lot in the future. It very well could cost nothing. Or it might be expensive enough to sink an enterprise. In the case of the original poster, it could well be that the maintenance he's doing to the code is minor and of no real cost, nor any maintenance burden in the future. He might not need security patches or any other on-going development of the Python 2 interpreter. His app might have no attack surface. The script does its internal job and that's that. As long as a Python2 interpreter runs he's golden. I suspect a lot of Python 2 use falls into that sort of category. Sort of like the company I know that still uses an MS-DOS billing system. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 12/02/2020 17:46, Python wrote: On Wed, Feb 12, 2020 at 01:16:03PM +, Rhodri James wrote: On 12/02/2020 00:53, Python wrote: In pretty much every job I've ever worked at, funding work (e.g. with humans to do it) with exactly and precisely the resources required is basically impossible, and management prefers to underfund the work than to overfund it, for cost-savings reasons. This basically means that any non-trivial work you do inevitably will become technical debt s/become/accrue/. The work itself isn't the debt, but its sub-optimality creates debt (or future headaches, if you prefer to think of it that way). I think it's a purely semantic distinction without a practical difference...which was the point I was trying to make. The work is the direct cause of the debt, and at the time it is performed the debt is realized. Without the work, that particular debt is not incurred. You may have eliminated some old debt when the work is done, but your new debt replaces your old debt. Depending on the resources you can devote, that debt may or MAY NOT be less than the other, and sometimes the truth of this can not be discovered until you're already knee deep in it. Here's where the "purely semantic" distinction matters. You are equating the work with the debt, but ignoring the benefit the work presumably brings (otherwise you wouldn't have done it at all). Either you should be rolling it all together or separate both, surely. -- Rhodri James *-* Kynesim Ltd -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 01:16:03PM +, Rhodri James wrote: > On 12/02/2020 00:53, Python wrote: > > In pretty much every job I've ever worked at, funding work (e.g. with > > humans to do it) with exactly and precisely the resources required is > > basically impossible, and management prefers to underfund the work > > than to overfund it, for cost-savings reasons. This basically means > > that any non-trivial work you do inevitably will become technical debt > > s/become/accrue/. The work itself isn't the debt, but its sub-optimality > creates debt (or future headaches, if you prefer to think of it that way). I think it's a purely semantic distinction without a practical difference...which was the point I was trying to make. The work is the direct cause of the debt, and at the time it is performed the debt is realized. Without the work, that particular debt is not incurred. You may have eliminated some old debt when the work is done, but your new debt replaces your old debt. Depending on the resources you can devote, that debt may or MAY NOT be less than the other, and sometimes the truth of this can not be discovered until you're already knee deep in it. > > So conceptually "costs" may be different from "debt" but in > > practice, you never have one without the other, and "debt" is > > really just "costs" you haven't paid yet. > > It's that last bit, "you haven't paid yet", that's the important part. > Project managers and accountants alike are very much in favour of putting > off paying for things if they can. Sometimes they can be persuaded that the > interest on the debt (the extra cost that the code structure imposes on > fixing bugs) is too much, and then you get the opportunity to refactor and > reduce the overall debt (at a cost, obviously) and the interest you will pay > in the future. Right. Or... not. More often than not, in my experience. And sometimes you can convince them you need to, but other priorities continually surface that block it from ever happening anyway. > Sometimes, and this is the bit that bean counters really like, you can get > away without paying the debt by ditching the project entirely before the > debt is paid off. If you don't want to piss your customers off you need to > pay the cost of a replacement project, which will accrue its own technical > debt... Or it may become obsolete due to changing circumstances. Or (as in many cases) it is sufficient to tell your customers, "don't do that," perhaps with admittedly annoying but not particularly costly consequences if they don't listen. :) [It's not always feasible, but often if they complain, you can say something like, "...but we're working on this other magic thingy that will really help you, and spending time on fixing this thing you're complaining about now will significantly delay the delivery of that." It may or may not be true, but in the interim the customer learns to stop doing that thing, and forgets they ever cared.] Technical debt is just about inevitable, for most non-trivial software (again, unless you've got a special case of a full solution for some problem whose circumstances never change), and isn't necessarily a bad thing. It just depends entirely on the details of your situation. And while I'm not particularly a fan of Agile in practice, the philosophy behind it addresses this reasonably well... But then again, you don't *need* Agile to do that, either. You just have to pay attention to the problem. Like everything else. -- https://mail.python.org/mailman/listinfo/python-list
RE: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
I have to wonder if this is a bit like what happens when something like Windows offers you an upgrade if you pay for it. Some people have noticed how after such things come out, a series of rapid bug fixes come along. So, they wait. Some wait long enough until another entire version has come along or even several such cycles. They then try to jump from say version 2.1 to version 5.0 and skip paying for all the in-betweens. Others may wait till something forces them to change like receiving documents their version of EXCEL cannot handle properly or at all. And, when that happens, they may simply jump to a different product entirely, like leaving Lotus for EXCEL. And, as noted, some simply move on. All kinds of calendar programs and contact list managers can become obsolete when it is all bundled into something like Microsoft office Outlook. At some point, you toss the old or use some tool to migrate your data, or start over. With your own software a migration can be hairy and especially when the original staff are long gone or promoted and the manager has no clue and no budget for this. There seems to be substantial risk versus just leaving it alone and hoping it works long enough for you to get your bonus or move on. It is worse if the people maintaining it sort of made their own kludge variations like reinventing new features on their own in ways not compatible with the new. Do you port the kludge or switch over to the new way even if it means restructuring other parts of the code. A serious question. What has happened in other aspects of the field where a big enough change bifurcates the community? I am wondering what happened when PERL made incompatible changes. Are people still using both versions? What about largely discontinued languages that stopped being developed in any way and do not even have a compiler for newer machines? I know Microsoft periodically declares no further support for much earlier versions of Windows. I bet people who have old machines keep running without an upgrade, especially when their machine does not support the new version because it does not have enough memory or whatever. Why keep the old? I am sure they have their reasons that boil down to it runs programs they know well and that do what they need and do not add many bells and whistles that they don't think they need or that force them to change their ways. For some touch typists, the earlier word processors that do not know about a mouse and run in one big window, may be just right. But try sending one to others without using some conversion method first. But anyone teaching Python today who still uses version 2 exclusively may have some explaining to do unless the goal is to maintain and migrate those using it to version 3. Are there any serious new projects being built NOW using version 2? -Original Message- From: Python-list On Behalf Of Rhodri James Sent: Wednesday, February 12, 2020 8:16 AM To: python-list@python.org Subject: Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them) On 12/02/2020 00:53, Python wrote: > In pretty much every job I've ever worked at, funding work (e.g. with > humans to do it) with exactly and precisely the resources required is > basically impossible, and management prefers to underfund the work > than to overfund it, for cost-savings reasons. This basically means > that any non-trivial work you do inevitably will become technical debt s/become/accrue/. The work itself isn't the debt, but its sub-optimality creates debt (or future headaches, if you prefer to think of it that way). > IMMEDIATELY, because you will not be given the time to do the job > completely in the first place, there will inevitably be bugs which are > minor enough to ignore indefinitely, and most likely, in order to meet > arbitrary-but-nevertheless-real time constraints you will find > yourself obliged to take shortcuts. So conceptually "costs" may be > different from "debt" but in practice, you never have one without the > other, and "debt" is really just "costs" you haven't paid yet. It's that last bit, "you haven't paid yet", that's the important part. Project managers and accountants alike are very much in favour of putting off paying for things if they can. Sometimes they can be persuaded that the interest on the debt (the extra cost that the code structure imposes on fixing bugs) is too much, and then you get the opportunity to refactor and reduce the overall debt (at a cost, obviously) and the interest you will pay in the future. Sometimes, and this is the bit that bean counters really like, you can get away without paying the debt by ditching the project entirely before the debt is paid off. If you don't want to piss your customers off you need to pay the cost of a replacement project, which will accrue i
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 02/11/2020 04:38 PM, Michael Torrie wrote: It's all just different ways of accounting for the same things. In the olden days before the term "technical debt" was invented, we called this "total cost of ownership." TCO is not a fixed number. For example, if a loan is taken to help fund a project, then the "interest debt" will be a portion of the TCO, but its amount will vary depending on the interest rate: 15% will be more interest debt than 4%. Likewise, the technical debt for a project will be higher or lower depending on the quality of the code written. I think an oft overlooked aspect of technical debt is the affect on the programmers dealing with it: frustration, burn-out, and job-seeking. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 12/02/2020 00:53, Python wrote: In pretty much every job I've ever worked at, funding work (e.g. with humans to do it) with exactly and precisely the resources required is basically impossible, and management prefers to underfund the work than to overfund it, for cost-savings reasons. This basically means that any non-trivial work you do inevitably will become technical debt s/become/accrue/. The work itself isn't the debt, but its sub-optimality creates debt (or future headaches, if you prefer to think of it that way). IMMEDIATELY, because you will not be given the time to do the job completely in the first place, there will inevitably be bugs which are minor enough to ignore indefinitely, and most likely, in order to meet arbitrary-but-nevertheless-real time constraints you will find yourself obliged to take shortcuts. So conceptually "costs" may be different from "debt" but in practice, you never have one without the other, and "debt" is really just "costs" you haven't paid yet. It's that last bit, "you haven't paid yet", that's the important part. Project managers and accountants alike are very much in favour of putting off paying for things if they can. Sometimes they can be persuaded that the interest on the debt (the extra cost that the code structure imposes on fixing bugs) is too much, and then you get the opportunity to refactor and reduce the overall debt (at a cost, obviously) and the interest you will pay in the future. Sometimes, and this is the bit that bean counters really like, you can get away without paying the debt by ditching the project entirely before the debt is paid off. If you don't want to piss your customers off you need to pay the cost of a replacement project, which will accrue its own technical debt... -- Rhodri James *-* Kynesim Ltd -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2020-02-12, Chris Angelico wrote: > But you CAN rewrite code such that it reduces technical debt. You can > refactor code to make it more logical. ... but if doing so costs more than the debt, you shouldn't do it. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 12:32 PM Michael Torrie wrote: > > On 2/11/20 6:15 PM, Chris Angelico wrote: > > On Wed, Feb 12, 2020 at 12:13 PM Michael Torrie wrote: > >> > >> On 2/11/20 5:55 PM, Chris Angelico wrote: > >>> But you CAN rewrite code such that it reduces technical debt. You can > >>> refactor code to make it more logical. You can update things to use > >>> idioms that better express the concepts you're trying to represent > >>> (maybe because those idioms require syntactic features that didn't > >>> exist, or simply because you didn't know about them when you first > >>> wrote the code). Maybe you'll still have SOME debt, but that doesn't > >>> mean it's never reduced. > >>> > >>> Debt is not a binary state. > >> > >> I agree with that. But your reply to my other comment didn't say that. > >> it said "it CAN be paid off" which is a binary thing. Debt is paid off > >> (no longer existing) or it's not. Debt can be paid down and reduced of > >> course. > > > > Ahh, that might be a regional difference then, because around here, > > it's possible to pay off some of a debt. That would be why we were > > talking past each other. > > Yup I just came to that conclusion while typing a reply to your other > post! My apologies. > My apologies also :) It was one of those weird situations where I was all "oh come on, he doesn't SERIOUSLY think that you never get rid of any tech debt?" and you were all "oh come on, he doesn't SERIOUSLY think that you can get rid of all of it", and we just both had no idea :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 6:15 PM, Chris Angelico wrote: > On Wed, Feb 12, 2020 at 12:13 PM Michael Torrie wrote: >> >> On 2/11/20 5:55 PM, Chris Angelico wrote: >>> But you CAN rewrite code such that it reduces technical debt. You can >>> refactor code to make it more logical. You can update things to use >>> idioms that better express the concepts you're trying to represent >>> (maybe because those idioms require syntactic features that didn't >>> exist, or simply because you didn't know about them when you first >>> wrote the code). Maybe you'll still have SOME debt, but that doesn't >>> mean it's never reduced. >>> >>> Debt is not a binary state. >> >> I agree with that. But your reply to my other comment didn't say that. >> it said "it CAN be paid off" which is a binary thing. Debt is paid off >> (no longer existing) or it's not. Debt can be paid down and reduced of >> course. > > Ahh, that might be a regional difference then, because around here, > it's possible to pay off some of a debt. That would be why we were > talking past each other. Yup I just came to that conclusion while typing a reply to your other post! My apologies. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 5:53 PM, Python wrote: > If your hypothetical project was implemented perfectly from the > beginning, in Python2.x, it may never need updating, and therefore > there may well never be any reason to port it to python3. So doing so > would be neither "debt" nor "cost" but rather "waste." I would agree generally, except eventually Python2 will be unavailable in your distro and may no longer be build-able on current OS's. However if the program is working as well as you state, then the cost of converting to the current version of Python (whatever that will be) is not going to increase significantly. Whatever the case, technical debt belongs to the entity that owns the code. I know some folks still running an MS-DOS system for billing. Thanks to things like DosBox, this system will live on for a long time yet. It works, and they don't have any real need for anything beyond it. It's never needed much updating or maintenance beyond maintenance they would do anyway like backups. Insane amounts of technical debt? Maybe. Or maybe just the cost of buying a new off-the-shelf system, which would be the same cost (or more likely a lot more with upgrades and service plans) had they done this years ago. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 12:13 PM Michael Torrie wrote: > > On 2/11/20 5:55 PM, Chris Angelico wrote: > > But you CAN rewrite code such that it reduces technical debt. You can > > refactor code to make it more logical. You can update things to use > > idioms that better express the concepts you're trying to represent > > (maybe because those idioms require syntactic features that didn't > > exist, or simply because you didn't know about them when you first > > wrote the code). Maybe you'll still have SOME debt, but that doesn't > > mean it's never reduced. > > > > Debt is not a binary state. > > I agree with that. But your reply to my other comment didn't say that. > it said "it CAN be paid off" which is a binary thing. Debt is paid off > (no longer existing) or it's not. Debt can be paid down and reduced of > course. Ahh, that might be a regional difference then, because around here, it's possible to pay off some of a debt. That would be why we were talking past each other. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 5:55 PM, Chris Angelico wrote: > But you CAN rewrite code such that it reduces technical debt. You can > refactor code to make it more logical. You can update things to use > idioms that better express the concepts you're trying to represent > (maybe because those idioms require syntactic features that didn't > exist, or simply because you didn't know about them when you first > wrote the code). Maybe you'll still have SOME debt, but that doesn't > mean it's never reduced. > > Debt is not a binary state. I agree with that. But your reply to my other comment didn't say that. it said "it CAN be paid off" which is a binary thing. Debt is paid off (no longer existing) or it's not. Debt can be paid down and reduced of course. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 12:00 PM Michael Torrie wrote: > > On 2/11/20 5:42 PM, Chris Angelico wrote: > > Yes, if you consider the term to be synonymous with TCO, then > > naturally you'll see it as useless. But it isn't. Technical debt is a > > very specific thing and it CAN be paid off. > > We'll agree to disagree on the last bit. And I'm not the only one that > believes technical debt can never be paid off. Microsoft got fabulously > wealthy incurring vast amounts of technical debt. We can argue that > Windows is the result (it is), but what's killing Windows isn't the > technical debt. It's the cloud and the web. So what you're showing is that, sometimes, technical debt isn't paid off. It's a bit of a logical leap to go from there to "technical debt CAN'T be paid off", don't you think? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/2020 3:09 PM, Chris Angelico wrote: What you're talking about is costs in general, but "debt" is a very specific term. You accrue technical debt whenever you "borrow" time from the future - doing something that's less effort now at the expense of being worse in the future. A prime example is sending code to production without automated tests. You pay off that debt when you sink time into something in order to make it easier to work on in the future. In May 2013, idlelib had no test suite, no test/test_idle.py, and a few non-unittest unit tests. Coverage is now somewhere around 50% and tested modules are much easier to work with. The most common form of technical debt is legacy code, where you often end up paying interest on the debt every time you dip your toes into the code to make a small change, avoiding the work of actually refactoring things and fixing the problems. Without automated tests, every little change required manual testing and carried a non-zero chance of a regression. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 11:48 AM Michael Torrie wrote: > > On 2/11/20 5:37 PM, Chris Angelico wrote: > > On Wed, Feb 12, 2020 at 11:32 AM Michael Torrie wrote: > >> > >> On 2/11/20 2:25 PM, Barry Scott wrote: > >>> At Chris said moving to python3 will *reduce* your technical debt. > >>> You are paying off the debt. > >> > >> While at the same time incurring new debt. > > > > That's not an intrinsic part of the rewrite, and will only happen if > > you do the job sloppily. > > > > Perhaps you're completely misunderstanding the meaning of the term? > > > > https://en.wikipedia.org/wiki/Technical_debt > > https://thedailywtf.com/articles/technical-debt > > Yes I understand the meaning. Getting code out the door now, at the > expense of maintenance later. But really all code is technical debt. > That's my main point. No one writes good enough code to be completely > free of this technical debt. But you CAN rewrite code such that it reduces technical debt. You can refactor code to make it more logical. You can update things to use idioms that better express the concepts you're trying to represent (maybe because those idioms require syntactic features that didn't exist, or simply because you didn't know about them when you first wrote the code). Maybe you'll still have SOME debt, but that doesn't mean it's never reduced. Debt is not a binary state. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 5:42 PM, Chris Angelico wrote: > Yes, if you consider the term to be synonymous with TCO, then > naturally you'll see it as useless. But it isn't. Technical debt is a > very specific thing and it CAN be paid off. We'll agree to disagree on the last bit. And I'm not the only one that believes technical debt can never be paid off. Microsoft got fabulously wealthy incurring vast amounts of technical debt. We can argue that Windows is the result (it is), but what's killing Windows isn't the technical debt. It's the cloud and the web. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 07:09:12AM +1100, Chris Angelico wrote: > On Wed, Feb 12, 2020 at 7:03 AM Michael Torrie wrote: > > Speaking about technical debt is certainly fashionable these days. As > > if we've somehow discovered a brand new way of looking at things. But > > it doesn't matter what you do, there's always real cost, and therefore > > always technical debt. Moving to Python 3 incurs technical debt. > > Staying with Python 2 incurs technical debt. Thus I wonder if the term > > is actually that useful. [...] > What you're talking about is costs in general, but "debt" is a very > specific term. You accrue technical debt whenever you "borrow" time > from the future - doing something that's less effort now at the > expense of being worse in the future. In pretty much every job I've ever worked at, funding work (e.g. with humans to do it) with exactly and precisely the resources required is basically impossible, and management prefers to underfund the work than to overfund it, for cost-savings reasons. This basically means that any non-trivial work you do inevitably will become technical debt IMMEDIATELY, because you will not be given the time to do the job completely in the first place, there will inevitably be bugs which are minor enough to ignore indefinitely, and most likely, in order to meet arbitrary-but-nevertheless-real time constraints you will find yourself obliged to take shortcuts. So conceptually "costs" may be different from "debt" but in practice, you never have one without the other, and "debt" is really just "costs" you haven't paid yet. If your hypothetical project was implemented perfectly from the beginning, in Python2.x, it may never need updating, and therefore there may well never be any reason to port it to python3. So doing so would be neither "debt" nor "cost" but rather "waste." -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 5:37 PM, Chris Angelico wrote: > On Wed, Feb 12, 2020 at 11:32 AM Michael Torrie wrote: >> >> On 2/11/20 2:25 PM, Barry Scott wrote: >>> At Chris said moving to python3 will *reduce* your technical debt. >>> You are paying off the debt. >> >> While at the same time incurring new debt. > > That's not an intrinsic part of the rewrite, and will only happen if > you do the job sloppily. > > Perhaps you're completely misunderstanding the meaning of the term? > > https://en.wikipedia.org/wiki/Technical_debt > https://thedailywtf.com/articles/technical-debt Yes I understand the meaning. Getting code out the door now, at the expense of maintenance later. But really all code is technical debt. That's my main point. No one writes good enough code to be completely free of this technical debt. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 11:39 AM Michael Torrie wrote: > > On 2/11/20 1:09 PM, Chris Angelico wrote: > > What you're talking about is costs in general, but "debt" is a very > > specific term. You accrue technical debt whenever you "borrow" time > > from the future - doing something that's less effort now at the > > expense of being worse in the future. You pay off that debt when you > > sink time into something in order to make it easier to work on in the > > future. The most common form of technical debt is legacy code, where > > you often end up paying interest on the debt every time you dip your > > toes into the code to make a small change, avoiding the work of > > actually refactoring things and fixing the problems. > > It's all just different ways of accounting for the same things. In the > olden days before the term "technical debt" was invented, we called this > "total cost of ownership." This not only included the up front cost, but > the on-going (and potentially increasing) cost of maintenance, and often > even the future cost of migrating to a new solution. So in the end it's > all the same: cost. And it's never paid off. Ever. That's why I've > recently come to question the usefulness of the term "technical debt." Yes, if you consider the term to be synonymous with TCO, then naturally you'll see it as useless. But it isn't. Technical debt is a very specific thing and it CAN be paid off. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 1:09 PM, Chris Angelico wrote: > What you're talking about is costs in general, but "debt" is a very > specific term. You accrue technical debt whenever you "borrow" time > from the future - doing something that's less effort now at the > expense of being worse in the future. You pay off that debt when you > sink time into something in order to make it easier to work on in the > future. The most common form of technical debt is legacy code, where > you often end up paying interest on the debt every time you dip your > toes into the code to make a small change, avoiding the work of > actually refactoring things and fixing the problems. It's all just different ways of accounting for the same things. In the olden days before the term "technical debt" was invented, we called this "total cost of ownership." This not only included the up front cost, but the on-going (and potentially increasing) cost of maintenance, and often even the future cost of migrating to a new solution. So in the end it's all the same: cost. And it's never paid off. Ever. That's why I've recently come to question the usefulness of the term "technical debt." > Porting to Python 3 should *improve* your codebase, so it should be a > way of shedding technical debt. (Unless you do it by running 2to3 on > your code and hoping for the best. But that's a bad idea for many > other reasons.) But see, this is the thing. There's brand new technical debt accrued with the move to Python 3. The cost of the debt is lower, but it's still there. Because maintenance is still going to cost. Versions still bump and require hopefully minor tweaks. Eventually the bigger jumps come. New ideas come along also. New frameworks, new paradigms. I think it's a fallacy to think we can pay down technical debt. I'm sure we probably disagree. -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 11:32 AM Michael Torrie wrote: > > On 2/11/20 2:25 PM, Barry Scott wrote: > > At Chris said moving to python3 will *reduce* your technical debt. > > You are paying off the debt. > > While at the same time incurring new debt. That's not an intrinsic part of the rewrite, and will only happen if you do the job sloppily. Perhaps you're completely misunderstanding the meaning of the term? https://en.wikipedia.org/wiki/Technical_debt https://thedailywtf.com/articles/technical-debt ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 2:25 PM, Barry Scott wrote: > At Chris said moving to python3 will *reduce* your technical debt. > You are paying off the debt. While at the same time incurring new debt. > Not to mention that its harder to hire people to work on tech-debt legacy > code. > > Given the choice between a legacy python2 job and a modern python3 job > what would you choose? If this was an income job, it would entirely depend on what it paid. In this day and age of the so-called "gig economy" it really doesn't matter. I likely wouldn't be around long enough to have to personally deal with the long-term consequences (pay the debt as you say). -- https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
> On 11 Feb 2020, at 20:01, Michael Torrie wrote: > > On 2/11/20 4:05 AM, Chris Angelico wrote: >> Or just the recognition that, eventually, technical debt has to be >> paid. > > Speaking about technical debt is certainly fashionable these days. As > if we've somehow discovered a brand new way of looking at things. But > it doesn't matter what you do, there's always real cost, and therefore > always technical debt. Moving to Python 3 incurs technical debt. > Staying with Python 2 incurs technical debt. Thus I wonder if the term > is actually that useful. At Chris said moving to python3 will *reduce* your technical debt. You are paying off the debt. > > I know what you mean, though. The cost of staying with Python2 is > increasing rapidly compared to the cost of porting to Python3. Unlike > the nebulous term, "technical debt," the cost of staying with Python2 vs > porting to Python3 can be quantified in real dollar amounts. I've no > doubt that the calculus is in favor of Python2 a while longer for many > people. Not to mention that its harder to hire people to work on tech-debt legacy code. Given the choice between a legacy python2 job and a modern python3 job what would you choose? Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
> On 10 Feb 2020, at 23:01, Python wrote:
>
> As best I can tell, Python has no means to make use of the system's
> timezone info. In order to make datetime "timezone aware", you need
> to manually create a subclass of datetime.tzinfo, whose methods return
> the correct values for the timezone you care about. In the general
> case, this is hard, but since the timezone my dates are in is always
> GMT, it's no problem:
>
> class GMT(datetime.tzinfo):
>def utcoffset(self, dt):
>return datetime.timedelta(hours=0)
>def dst(self, dt):
>return datetime.timedelta(minutes=0)
>def tzname(self, dt):
>return "GMT"
>
> Now, you can instantiate a datetime.datetime object with the times you
> want, and pass an instance of this class as the tzinfo argument to the
> constructor. Also no problem:
dt = datetime.datetime(2020, 1, 31, 1, 30, 45, 987654, GMT())
dt
> datetime.datetime(2020, 1, 31, 1, 30, 45, 987654, tzinfo=<__main__.GMT object
> at 0x7f9084e2add0>)
print dt
> 2020-01-31 01:30:45.987654+00:00
>
> Note the tzinfo object, and the +00:00 indicating this is indeed in
> GMT. If you create a "naive" datetime object, you don't get that:
>
xy = datetime.datetime(2020, 1, 31, 1, 30, 45, 987654)
xy
> datetime.datetime(2020, 1, 31, 1, 30, 45, 987654)
print xy
> 2020-01-31 01:30:45.987654
>
>
> So far, so good. However, when you go to use this object, the time it
> represents is in fact wrong. For example:
>
print dt.strftime("%s")
> 1580452245
>
> Using the date command, we can easily see that it is incorrect:
>
> # Ask for UTC, gives the wrong time
> $ date -u -d "@1580452245"
> Fri Jan 31 06:30:45 UTC 2020
>
> # Ask for local time, gives the time I specified... which should have
> # been in GMT, but is in my local timezone
> $ date -d "@1580452245"
> Fri Jan 31 01:30:45 EST 2020
>
> And the correct value should have been:
> $ date -d "2020-01-31 1:30:45 GMT" +%s
> 1580434245
>
> Same results under Python2.7 or Python3.
>
> :( :( :(
>
> I suspect this is because the underlying implementation uses
> strftime() which takes struct tm, which has no field to contain the
> timezone info, and does not do the necessary conversion before handing
> it off. I'm not sure what the date command is doing differently, but
> it clearly is able to get the correct answer.
>
> Does Python have an alternative way to do the above, with correct
> results?
I found that there are two useful PyPi packages to help with time zones.
pytz and tzlocal.
Here is a piece of code that I use to be timezone aware for UTC and the
local time zone. It works for Windows, macOS and unix thanks.
import datetime
import pytz
import tzlocal
def utcDatetime( timestamp ):
return pytz.utc.localize( datetime.datetime.utcfromtimestamp( timestamp ) )
def localDatetime( datetime_or_timestamp ):
if type(datetime_or_timestamp) in (int, float):
dt = utcDatetime( datetime_or_timestamp )
else:
dt = datetime_or_timestamp
local_timezone = tzlocal.get_localzone()
local_dt = dt.astimezone( local_timezone )
return local_dt
Barry
>
> Thanks
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 7:03 AM Michael Torrie wrote: > > On 2/11/20 4:05 AM, Chris Angelico wrote: > > Or just the recognition that, eventually, technical debt has to be > > paid. > > Speaking about technical debt is certainly fashionable these days. As > if we've somehow discovered a brand new way of looking at things. But > it doesn't matter what you do, there's always real cost, and therefore > always technical debt. Moving to Python 3 incurs technical debt. > Staying with Python 2 incurs technical debt. Thus I wonder if the term > is actually that useful. > > I know what you mean, though. The cost of staying with Python2 is > increasing rapidly compared to the cost of porting to Python3. Unlike > the nebulous term, "technical debt," the cost of staying with Python2 vs > porting to Python3 can be quantified in real dollar amounts. I've no > doubt that the calculus is in favor of Python2 a while longer for many > people. What you're talking about is costs in general, but "debt" is a very specific term. You accrue technical debt whenever you "borrow" time from the future - doing something that's less effort now at the expense of being worse in the future. You pay off that debt when you sink time into something in order to make it easier to work on in the future. The most common form of technical debt is legacy code, where you often end up paying interest on the debt every time you dip your toes into the code to make a small change, avoiding the work of actually refactoring things and fixing the problems. Porting to Python 3 should *improve* your codebase, so it should be a way of shedding technical debt. (Unless you do it by running 2to3 on your code and hoping for the best. But that's a bad idea for many other reasons.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 6:45 AM Python wrote: > > On Wed, Feb 12, 2020 at 05:38:38AM +1100, Chris Angelico wrote: > > > > Isn't it time to stop going to great effort to support Python > > > > > > If you live in a world where you get to decide that, sure. Not > > > everyone does. > > > > > > > Everyone gets to decide how much time and effort they put into > > supporting an old and unsupported system. > > Mostly not... In the real world you typically are beholden to > customers (be they internal or external), and you are beholden to your > bosses (be they the same or different people from the first group). > What you say is true only if you have no customers or bosses to whom > you have obligations, or have sufficient financial freedom or > influence to ignore them and are willing to accept the consequences. > Is your boss/client smart enough to understand the concept of putting in effort now to pay off significant dividends later? Does your boss/client recognize that using software that isn't getting security patches means you're vulnerable? Actually, does your boss/client even care what your chosen language is? Certainly in the case of clients or customers, it's extremely common for them to have expectations in terms of "make it work", not "keep it running on Python 2". So long as it's fairly easy to keep it going on Py2, sure, keep it going on Py2. But if it's going to be a lot of effort - and, increasingly, it will - then you need to be willing to take the non-lazy option and do the migration, rather than perpetually putting it off and incurring more and more technical debt in the process. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Technical debt - was Re: datetime seems to be broken WRT timezones (even when you add them)
On 2/11/20 4:05 AM, Chris Angelico wrote: > Or just the recognition that, eventually, technical debt has to be > paid. Speaking about technical debt is certainly fashionable these days. As if we've somehow discovered a brand new way of looking at things. But it doesn't matter what you do, there's always real cost, and therefore always technical debt. Moving to Python 3 incurs technical debt. Staying with Python 2 incurs technical debt. Thus I wonder if the term is actually that useful. I know what you mean, though. The cost of staying with Python2 is increasing rapidly compared to the cost of porting to Python3. Unlike the nebulous term, "technical debt," the cost of staying with Python2 vs porting to Python3 can be quantified in real dollar amounts. I've no doubt that the calculus is in favor of Python2 a while longer for many people. -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 05:38:38AM +1100, Chris Angelico wrote: > > > Isn't it time to stop going to great effort to support Python > > > > If you live in a world where you get to decide that, sure. Not > > everyone does. > > > > Everyone gets to decide how much time and effort they put into > supporting an old and unsupported system. Mostly not... In the real world you typically are beholden to customers (be they internal or external), and you are beholden to your bosses (be they the same or different people from the first group). What you say is true only if you have no customers or bosses to whom you have obligations, or have sufficient financial freedom or influence to ignore them and are willing to accept the consequences. -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Wed, Feb 12, 2020 at 3:57 AM Python wrote: > > On Tue, Feb 11, 2020 at 12:42:26PM +1100, Chris Angelico wrote: > > > > I've been using the timestamp() method: > > > > > > That's the key piece of info. This does appear to work, though still > > > not on python2. That, as you say, is my problem. But thankfully Jon > > > Ribbens has the save: > > > > Isn't it time to stop going to great effort to support Python > > If you live in a world where you get to decide that, sure. Not > everyone does. > Everyone gets to decide how much time and effort they put into supporting an old and unsupported system. Are you telling me that someone is enslaving you and forcing you to continue supporting Python 2, or are you actually saying that it's less effort to keep maintaining old code than to port to Py3? As I said in the previous email, I did NOT say that it's time to instantly drop Py2 like a hot potato. (And if you do nothing, a hot potato just becomes a cold potato. Thank you, Bernard.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 12:42:26PM +1100, Chris Angelico wrote: > > > I've been using the timestamp() method: > > > > That's the key piece of info. This does appear to work, though still > > not on python2. That, as you say, is my problem. But thankfully Jon > > Ribbens has the save: > > Isn't it time to stop going to great effort to support Python If you live in a world where you get to decide that, sure. Not everyone does. -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On 2020-02-11, Chris Angelico wrote: > On Tue, Feb 11, 2020 at 10:01 PM Jon Ribbens via Python-list > wrote: >> So while it's been about 6 years since anyone should have been >> starting any new projects using Python 2, there are plenty of >> projects that are older than that and still need supporting, >> and often it'd take some pretty huge unavoidable requirement >> to motivate a port to Python 3. > > Or just the recognition that, eventually, technical debt has to be > paid. I didn't say that everything has to stop supporting Py2 > instantly now that it's 2020, but that it's time to stop going to > great lengths for it. Py2 is a legacy system and has been for a long > time now, and if it takes a lot of effort to keep maintaining it, then > it's time to consider porting to Py3. Just how much work are you going > to do, to avoid the work of porting? Not doing something generally doesn't involve any work. Occasionally there's a very small amount of work when something that would have been trivial in Python 3 is slightly harder in Python 2 - this thread for example, where it appears to involve one extra line of code. -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 10:01 PM Jon Ribbens via Python-list wrote: > > On 2020-02-11, Chris Angelico wrote: > >> That's the key piece of info. This does appear to work, though still > >> not on python2. That, as you say, is my problem. But thankfully Jon > >> Ribbens has the save: > > > > Isn't it time to stop going to great effort to support Python 2? > > That depends on what existing systems people need to support. The main > project I work on pre-dates the existence of Python's 'datetime' > module, let alone Python 3. It still generally uses 1 and 0 for True > and False because True and False didn't exist when we started. > > This project will never move to Python 3. It was a happy and momentous > day, fairly recently, when we were able to move to Python 2.7 - albeit > we're basically stuck on Python 2.7.5 (plus random patches from RedHat > making it not-really-2.7.5). > > So while it's been about 6 years since anyone should have been > starting any new projects using Python 2, there are plenty of > projects that are older than that and still need supporting, > and often it'd take some pretty huge unavoidable requirement > to motivate a port to Python 3. Or just the recognition that, eventually, technical debt has to be paid. I didn't say that everything has to stop supporting Py2 instantly now that it's 2020, but that it's time to stop going to great lengths for it. Py2 is a legacy system and has been for a long time now, and if it takes a lot of effort to keep maintaining it, then it's time to consider porting to Py3. Just how much work are you going to do, to avoid the work of porting? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On 2020-02-11, Chris Angelico wrote: >> That's the key piece of info. This does appear to work, though still >> not on python2. That, as you say, is my problem. But thankfully Jon >> Ribbens has the save: > > Isn't it time to stop going to great effort to support Python 2? That depends on what existing systems people need to support. The main project I work on pre-dates the existence of Python's 'datetime' module, let alone Python 3. It still generally uses 1 and 0 for True and False because True and False didn't exist when we started. This project will never move to Python 3. It was a happy and momentous day, fairly recently, when we were able to move to Python 2.7 - albeit we're basically stuck on Python 2.7.5 (plus random patches from RedHat making it not-really-2.7.5). So while it's been about 6 years since anyone should have been starting any new projects using Python 2, there are plenty of projects that are older than that and still need supporting, and often it'd take some pretty huge unavoidable requirement to motivate a port to Python 3. -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 12:31 PM Python wrote:
>
> On Tue, Feb 11, 2020 at 11:33:54AM +1100, Chris Angelico wrote:
> > [...] instead of using the undocumented and unsupported strftime %s
> > format code
>
> I'm not sure this characterization is accurate... the docs (for both
> Python 2 and 3) say:
>
> The full set of format codes supported varies across platforms,
> because Python calls the platform C library’s strftime() function,
> and platform variations are common. To see the full set of format
> codes supported on your platform, consult the strftime(3)
> documentation.
>
> And %s is surely there... so it is both documented and supported by
> virtue of Python's docs saying that whatever your OS supports is
> supported. But this is largely beside the point.
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
The C standard mandates a particular set of format codes, quoted in
that page, and %s is not one of them. Also, Python has its own test
suite and it ensures certain behaviours of datetime.strftime, and %s
is not tested there. So from Python's point of view, I stand by the
description of %s as being undocumented and unsupported. It might
happen to work, but if you link Python against a different C standard
library, it could change that behaviour even on the same computer.
> > I've been using the timestamp() method:
>
> That's the key piece of info. This does appear to work, though still
> not on python2. That, as you say, is my problem. But thankfully Jon
> Ribbens has the save:
Isn't it time to stop going to great effort to support Python 2?
> On Tue, Feb 11, 2020 at 12:59:04AM -, Jon Ribbens via Python-list wrote:
> > On 2020-02-10, Python wrote:
> > > So far, so good. However, when you go to use this object, the time it
> > > represents is in fact wrong.
> >
> > Unsurprisingly for a language feature that's been around for nearly
> > 17 years, no it isn't.
>
> Well, fine, but it is at best not documented as clearly as it could be
> (in the man pages of my system, which is not Python's fault). But
> you've given me what I need to solve my problem:
>
> > print dt.strftime("%s")
> > > 1580452245
> >
> > That's asking your libc's strftime to process the time through the
> > "%s" format code [...] If you're using GNU libc however then it uses the
> > time
> > information given and the timezone from the 'TZ' environment variable
> > and returns seconds since the Unix epoch:
>
> And knowing this, I can get what I need, on Python2 and python3, by
> doing what I was already doing, but setting os.environ["TZ"] = "GMT"
> in the code itself. Sure it's a bit hacky, but it won't matter here,
> and it gets the right answer, which does matter.
>
Yes, it's very hacky. The sort of hack that deserves a comment with a
TODO in it. :)
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 11:33:54AM +1100, Chris Angelico wrote:
> [...] instead of using the undocumented and unsupported strftime %s
> format code
I'm not sure this characterization is accurate... the docs (for both
Python 2 and 3) say:
The full set of format codes supported varies across platforms,
because Python calls the platform C library’s strftime() function,
and platform variations are common. To see the full set of format
codes supported on your platform, consult the strftime(3)
documentation.
And %s is surely there... so it is both documented and supported by
virtue of Python's docs saying that whatever your OS supports is
supported. But this is largely beside the point.
> I've been using the timestamp() method:
That's the key piece of info. This does appear to work, though still
not on python2. That, as you say, is my problem. But thankfully Jon
Ribbens has the save:
On Tue, Feb 11, 2020 at 12:59:04AM -, Jon Ribbens via Python-list wrote:
> On 2020-02-10, Python wrote:
> > So far, so good. However, when you go to use this object, the time it
> > represents is in fact wrong.
>
> Unsurprisingly for a language feature that's been around for nearly
> 17 years, no it isn't.
Well, fine, but it is at best not documented as clearly as it could be
(in the man pages of my system, which is not Python's fault). But
you've given me what I need to solve my problem:
> print dt.strftime("%s")
> > 1580452245
>
> That's asking your libc's strftime to process the time through the
> "%s" format code [...] If you're using GNU libc however then it uses the time
> information given and the timezone from the 'TZ' environment variable
> and returns seconds since the Unix epoch:
And knowing this, I can get what I need, on Python2 and python3, by
doing what I was already doing, but setting os.environ["TZ"] = "GMT"
in the code itself. Sure it's a bit hacky, but it won't matter here,
and it gets the right answer, which does matter.
Thanks
--
https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On 2020-02-10, Python wrote:
> So far, so good. However, when you go to use this object, the time it
> represents is in fact wrong.
Unsurprisingly for a language feature that's been around for nearly
17 years, no it isn't.
> For example:
>
print dt.strftime("%s")
> 1580452245
That's asking your libc's strftime to process the time through the
"%s" format code, which has no defined behaviour under either Python
or POSIX. If you're using GNU libc however then it uses the time
information given and the timezone from the 'TZ' environment variable
and returns seconds since the Unix epoch:
>>> dt.strftime('%s')
'1580452245'
>>> import os
>>> os.environ['TZ'] = 'UTC'
>>> dt.strftime('%s')
'1580434245'
If you need to know the seconds since the epoch then in Python 3.3+
you can do that with dt.timestamp(). In Python 2.7 you can use the UTC
class you created:
(dt - datetime(1970, 1, 1, 0, 0, 0, 0, GMT())).total_seconds()
--
https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 11:17 AM Python wrote:
>
> On Tue, Feb 11, 2020 at 11:04:28AM +1100, Chris Angelico wrote:
> > On Tue, Feb 11, 2020 at 10:42 AM Python wrote:
> > > Now, you can instantiate a datetime.datetime object with the times you
> > > want, and pass an instance of this class as the tzinfo argument to the
> > > constructor. Also no problem:
> >
> > Rather than try to create your own GMT() object, have you considered
> > using datetime.timezone.utc ? I tested it in your examples and it
> > works just fine.
>
> Not here it doesn't:
>
> >>> import datetime
> >>> dt = datetime.datetime(2020, 1, 31, 1, 30, 45, 987654,
> >>> datetime.timezone.utc)
> >>> print(dt)
> 2020-01-31 01:30:45.987654+00:00
> >>> print(dt.strftime("%s"))
> 1580452245
>
> That's the same erroneous result from my example. The correct value
> is again 1580434245. Unless you've done something subtlely
> different...
I haven't verified the arithmetic or that the date command is actually
giving the result you want/expect here, but in my testing, I got what
looked like correct results. However, instead of using the
undocumented and unsupported strftime %s format code, I've been using
the timestamp() method:
https://docs.python.org/3/library/datetime.html#datetime.datetime.timestamp
rosuav@sikorsky:~$ cat utctest.py
import datetime
dt = datetime.datetime(2020, 1, 31, 1, 30, 45, 987654, datetime.timezone.utc)
print(dt)
print(dt.timestamp())
rosuav@sikorsky:~$ python3.9 utctest.py
2020-01-31 01:30:45.987654+00:00
1580434245.987654
rosuav@sikorsky:~$ python3.8 utctest.py
2020-01-31 01:30:45.987654+00:00
1580434245.987654
rosuav@sikorsky:~$ python3.7 utctest.py
2020-01-31 01:30:45.987654+00:00
1580434245.987654
rosuav@sikorsky:~$ python3.6 utctest.py
2020-01-31 01:30:45.987654+00:00
1580434245.987654
rosuav@sikorsky:~$ python3.5 utctest.py
2020-01-31 01:30:45.987654+00:00
1580434245.987654
rosuav@sikorsky:~$ python3.4 utctest.py
2020-01-31 01:30:45.987654+00:00
1580434245.987654
My interpretation is that, on your platform, strftime("%s") is broken
in the presence of tzinfo.
> Also it's only available on Python3, which may not be the end of the
> world, but the tool I'm working on is--for the moment at
> least--expected to work on both python2.7 and 3. But it's irrelevant
> if it doesn't give the correct result, which seems to still be the
> case.
>
That may be your problem, but it's not mine, and I'm not going to
attempt to answer for Python 2.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 11:04:28AM +1100, Chris Angelico wrote:
> On Tue, Feb 11, 2020 at 10:42 AM Python wrote:
> > Now, you can instantiate a datetime.datetime object with the times you
> > want, and pass an instance of this class as the tzinfo argument to the
> > constructor. Also no problem:
>
> Rather than try to create your own GMT() object, have you considered
> using datetime.timezone.utc ? I tested it in your examples and it
> works just fine.
Not here it doesn't:
>>> import datetime
>>> dt = datetime.datetime(2020, 1, 31, 1, 30, 45, 987654,
>>> datetime.timezone.utc)
>>> print(dt)
2020-01-31 01:30:45.987654+00:00
>>> print(dt.strftime("%s"))
1580452245
That's the same erroneous result from my example. The correct value
is again 1580434245. Unless you've done something subtlely
different...
Also it's only available on Python3, which may not be the end of the
world, but the tool I'm working on is--for the moment at
least--expected to work on both python2.7 and 3. But it's irrelevant
if it doesn't give the correct result, which seems to still be the
case.
--
https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Tue, Feb 11, 2020 at 10:42 AM Python wrote: > > As best I can tell, Python has no means to make use of the system's > timezone info. In order to make datetime "timezone aware", you need > to manually create a subclass of datetime.tzinfo, whose methods return > the correct values for the timezone you care about. In the general > case, this is hard, but since the timezone my dates are in is always > GMT, it's no problem: > > class GMT(datetime.tzinfo): > def utcoffset(self, dt): > return datetime.timedelta(hours=0) > def dst(self, dt): > return datetime.timedelta(minutes=0) > def tzname(self, dt): > return "GMT" > > Now, you can instantiate a datetime.datetime object with the times you > want, and pass an instance of this class as the tzinfo argument to the > constructor. Also no problem: Rather than try to create your own GMT() object, have you considered using datetime.timezone.utc ? I tested it in your examples and it works just fine. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
On Mon, Feb 10, 2020 at 05:52:59PM -0600, Skip Montanaro wrote: > > As best I can tell, Python has no means to make use of the system's > > timezone info. In order to make datetime "timezone aware", you need > > to manually create a subclass of datetime.tzinfo, whose methods return > > the correct values for the timezone you care about. > > > ... > > > Does Python have an alternative way to do the above, with correct > > results? > > > > The datetime module's manipulation of timezones is pretty low-level. Maybe > try arrow? > > https://arrow.readthedocs.io/en/latest/ Yeah, this looks like exacly what I'm looking for, but I need something that's part of the standard Python distribution, due to constraints I have no control over. [This is perhaps not strictly true, but for all intents and purposes...] FWIW, manipulating dates on POSIX systems is pretty much a nightmare, so it's hard to blame Python entirely for the state of things. Thanks though. -- https://mail.python.org/mailman/listinfo/python-list
Re: datetime seems to be broken WRT timezones (even when you add them)
> As best I can tell, Python has no means to make use of the system's > timezone info. In order to make datetime "timezone aware", you need > to manually create a subclass of datetime.tzinfo, whose methods return > the correct values for the timezone you care about. > ... > Does Python have an alternative way to do the above, with correct > results? > The datetime module's manipulation of timezones is pretty low-level. Maybe try arrow? https://arrow.readthedocs.io/en/latest/ Skip > -- https://mail.python.org/mailman/listinfo/python-list
